nsfw image detector

提供商Freepik
分类image-classification
许可证mit
下载量594.7K
星标0

简介

这款由 Freepik 提供的 NSFW 图像检测模型,旨在帮助开发者快速构建内容审核机制。它专注于识别图像中的非安全内容,将复杂的图像分类简化为简单的标签判定。对于需要处理用户上传图片、构建社区相册或管理素材库的中国开发者来说,这是一个低门槛的轻量化方案。该模型采用 MIT 许可,集成难度低,无需复杂的调优即可在应用端实现基础的视觉过滤,有效降低人工审核的成本。

核心亮点

  • 快速识别图像非安全内容,自动化审核
  • MIT 协议开源,商业集成灵活且无压力
  • 轻量级分类模型,部署简单,上手极快
  • 适用于社区 UGC 过滤与素材库质量控制

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("Freepik/nsfw_image_detector")
tokenizer = AutoTokenizer.from_pretrained("Freepik/nsfw_image_detector")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引:在下载前,请先通过如下命令安装 huggingface_hub:

操作指引
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库
huggingface-cli download Freepik/nsfw_image_detector

下载单个文件到指定本地文件夹(以下载 config.json 到当前路径下 ./dir 目录为例)

下载单个文件到指定本地文件夹(以下载 config.json 到当前路径下 ./dir 目录为例)
huggingface-cli download Freepik/nsfw_image_detector config.json --local-dir ./dir

更多命令行下载选项,可参见官方文档

SDK 下载

SDK 下载
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('Freepik/nsfw_image_detector')

Git 下载

请确保 lfs 已经被正确安装

Git 下载
git lfs install
git clone https://huggingface.co/Freepik/nsfw_image_detector

如果您希望跳过 lfs 大文件下载,可以使用如下命令

跳过 LFS
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Freepik/nsfw_image_detector

模型文件托管在 Hugging Face Hub,使用 HF CLI / SDK / Git 直接下载,不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers
pip install -U transformers torch

模型加载和推理

模型加载和推理
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('Freepik/nsfw_image_detector')
tokenizer = AutoTokenizer.from_pretrained('Freepik/nsfw_image_detector')

完整文档

来源: HuggingFace

---
license: mit
base_model:

  • timm/eva02_base_patch14_448.mim_in22k_ft_in22k_in1k

pipeline_tag: image-classification
tags:
  • pytorch

  • transformers

---

EVA-based Fast NSFW Image Classifier

Table of Contents

- Global Performance - Accuracy by AI Content - AI-Generated Content - Non-AI-Generated Content - Quick Start via pip - Quick Start with Pipeline - Avoid installation of pip dependency

Model Description

This model is a vision transformer based on the EVA architecture, fine-tuned for NSFW content classification. It has been trained
to detect four categories (neutral, low, medium, high) of visual content using 100,000 synthetically labeled images.

The model can be used as a binary (true/false) classifier if desired, or you can obtain the full output probabilities.. It outperforms other excellent publicly available models such as Falconsai/nsfw_image_detection or AdamCodd/vit-base-nsfw-detector in our internal benchmarks adding the enrichment of being able to select the NSFW level that suits your use case.

Try it Online! 🚀

You can try this model directly in your browser through our Hugging Face Space. Upload any image and get instant NSFW classification results without any installation required.

Model Performance Comparison

Global Performance

| Category | Freepik | Falconsai | Adamcodd |
|----------|-------------|------------------|----------------|
| High | 99.54% | 97.92% | 98.62% |
| Medium | 97.02% | 78.54% | 91.65% |
| Low | 98.31% | 31.25% | 89.66% |
| Neutral | 99.87% | 99.27% | 98.37% |

In the table below, the results are obtained as follows:

  • For the Falconsai and AdamCodd models:

* A prediction is considered correct if the image is labeled "low", "medium", or "high" and the model returns true.
* If the label is "neutral", the correct output should be false.

  • For the Freepik model:
* If the image label is "low", "medium", or "high", the model should return at least "low". * If the label is "neutral", the correct output should be "neutral".

Conclusions:

  • Our model outperforms AdamCodd and Falconsai in accuracy. It is entirely fair to compare them on the "high" and "neutral" labels.
  • Our model offers greater granularity. It is not only suitable for detecting "high" and "neutral" content, but also performs excellently at identifying "low" and "medium" NSFW content.
* Falconsai may classify some "medium" and "low" images as not NSFW but mark others as safe for work(SFW), which could lead to unexpected results. * AdamCodd classifies both "low" and "medium" categories as NSFW, which may not be desirable depending on your use case. Furthermore, a 10% of images in low and medium are considered SFW.

Accuracy by AI Content

We have created a manually labeled dataset with careful attention to avoiding biases (gender, ethnicity, etc.). While the sample size is relatively small, it provides meaningful insights into model performance across different scenarios, which was very useful in the training process to avoid biases.

The following tables show detection accuracy percentages across different NSFW categories and content types:

#### AI-Generated Content

| Category | Freepik Model | Falconsai Model | Adamcodd Model |
|----------|-------------|------------------|----------------|
| High | 100.00% | 84.00% | 92.00% |
| Medium | 96.15% | 69.23% | 96.00% |
| Low | 100.00% | 35.71% | 92.86% |
| Neutral | 100.00% | 100.00% | 66.67% |

Conclusions:

  • Avoid using Falconsai for AI-generated content to prevent prediction errors.

  • Our model is the best option to detect NSFW content in AI-generated content.

Usage

Quick Start via pip

```sh
pip install nsfw-image-detector

code
python
from PIL import Image
from nsfw_image_detector import NSFWDetector
import torch

Initialize the detector

detector = NSFWDetector(dtype=torch.bfloat16, device="cuda")

Load and classify an image

image = Image.open("your_image")

Check if the image contains NSFW content sentivity level medium or higher

is_nsfw = detector.is_nsfw(image, "medium")

Get probability scores for all categories

probabilities = detector.predict_proba(image) print(f"Is NSFW: {is_nsfw}") print(f"Probabilities: {probabilities}")
code
Example output:
python Is NSFW: False Probabilities: [ {<NSFWLevel.HIGH: 'high'>: 0.00372314453125, <NSFWLevel.MEDIUM: 'medium'>: 0.1884765625, <NSFWLevel.LOW: 'low'>: 0.234375, <NSFWLevel.NEUTRAL: 'neutral'>: 0.765625} ]
code
### Quick Start with Pipeline
python from transformers import pipeline from PIL import Image

Create classifier pipeline

classifier = pipeline( "image-classification", model="Freepik/nsfw_image_detector", device=0 # Use GPU (0) or CPU (-1) )

Load and classify an image

image = Image.open("path/to/your/image.jpg") predictions = classifier(image) print(predictions)
code
Example output:
python [ {'label': 'neutral', 'score': 0.92}, {'label': 'low', 'score': 0.05}, {'label': 'medium', 'score': 0.02}, {'label': 'high', 'score': 0.01} ]
code
The model supports efficient batch processing for multiple images:
python images = [Image.open(path) for path in ["image1.jpg", "image2.jpg", "image3.jpg"]] predictions = classifier(images)
code
Note: If the intention is to use the model in production review Speed and Memory Metrics section before using this approach.

Avoid installation of pip dependency

The following example demonstrates how to customize the NSFW detection label, it is very similar to the code in PyPy. This code returns True if the NSFW level is 'medium' or higher:

python
from transformers import AutoModelForImageClassification
import torch
from PIL import Image
from typing import List, Dict
import torch.nn.functional as F
from timm.data.transforms_factory import create_transform
from torchvision.transforms import Compose
from timm.data import resolve_data_config
from timm.models import get_pretrained_cfg

device = "cuda" if torch.cuda.is_available() else "cpu"

Load model and processor

model = AutoModelForImageClassification.from_pretrained("Freepik/nsfw_image_detector", torch_dtype = torch.bfloat16).to(device)

Load original processor (faster for tensors)

cfg = get_pretrained_cfg("eva02_base_patch14_448.mim_in22k_ft_in22k_in1k") processor: Compose = create_transform(**resolve_data_config(cfg.__dict__))

def predict_batch_values(model, processor: Compose, img_batch: List[Image.Image] | torch.Tensor) -> List[Dict[str, float]]:
"""
Process a batch of images and return prediction scores for each NSFW category
"""
idx_to_label = {0: 'neutral', 1: 'low', 2: 'medium', 3: 'high'}

# Prepare batch
inputs = torch.stack([processor(img) for img in img_batch])
output = []
with torch.inference_mode():
logits = model(inputs).logits
batch_probs = F.log_softmax(logits, dim=-1)
batch_probs = torch.exp(batch_probs).cpu()

for i in r