generated image quality assessment

提供商arman-chopikyan
分类image-quality-assessment
许可证Apache-2.0
下载量0
星标0

简介

这是一个专注于生成图像质量评估的轻量化模型，旨在解决 AI 绘图（如 Stable Diffusion 或 Midjourney）中常见的伪影、失真等质量判定问题。与传统的图像识别模型不同，它更像是一个“数字质检员”，能将图像的视觉质量量化为具体得分。对于需要大规模筛选生成结果、优化 Prompt 效果或构建自动化图像过滤管线的开发者来说，该模型提供了低门槛的集成方案，无需复杂配置即可快速评估图像的可用性。

核心亮点

  • 量化评估 AI 生成图质量，替代主观肉眼筛选
  • 适用于 AIGC 自动化工作流中的图像质检环节
  • Apache-2.0 协议，支持商业化部署与二次开发
  • 轻量化设计，可快速集成至图像处理管线

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("arman-chopikyan/generated-image-quality-assessment")
tokenizer = AutoTokenizer.from_pretrained("arman-chopikyan/generated-image-quality-assessment")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download arman-chopikyan/generated-image-quality-assessment

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download arman-chopikyan/generated-image-quality-assessment config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('arman-chopikyan/generated-image-quality-assessment')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/arman-chopikyan/generated-image-quality-assessment

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/arman-chopikyan/generated-image-quality-assessment

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('arman-chopikyan/generated-image-quality-assessment')
tokenizer = AutoTokenizer.from_pretrained('arman-chopikyan/generated-image-quality-assessment')

完整文档

来源: HuggingFace

Report: Generated Image Quality Assessment Model

1. Introduction

This report details the development of a model for evaluating the quality of AI-generated human images. The evaluation criteria include:

  • Relevance to the text prompt
  • Visual quality and aesthetics
  • Presence of artifacts and distortions
  • Overall Quality

The workflow consists of three key stages:

1. Dataset Preparation using ChatGPT VLM and CLIP embeddings.
2. Model Training of distilled model leveraging CLIP features.
3. Evaluation of model performance.

---

2. Dataset Preparation (Dataset.ipynb)

Due to cost and resource constraints, only 10,000 images were sourced from DiffusionDB, filtered to retain only human-related images. After removing invalid or problematic images, the final dataset size was 9,432.

Data Processing Workflow

1. Filtering Prompts: Extracting images with human-related keywords (e.g., "man", "woman", "child").
2. Downloading Images: Retrieving images from DiffusionDB based on filtered metadata.
3. Generating Annotations Using ChatGPT VLM:
- GPT-4o Mini was used to assess:
- Relevance to the prompt (0.0 - 1.0 scale)
- Visual quality (composition, clarity, aesthetics)
- Presence of artifacts (missing parts, distortions, unnatural anatomy). Important note: *1 indicates artifact presence, 0 indicates absence*. This approach was chosen as detecting artifact presence is significantly easier for the network compared to detecting their absence.
4. Generating CLIP Embeddings:
- Image and text embeddings were extracted using CLIP ViT-B/32.
5. Dataset Splitting:
- 80% training, 15% validation, and 5% test split.

Dataset Analysis

A histogram-based analysis was performed to examine score distributions. The dataset exhibited high imbalance, with some quality scores appearing far more frequently than others. To address this, weighted binary cross-entropy was employed during training.

The final dataset used in this study can be accessed at:

---

3. Model Training (Training.ipynb)

A lightweight regression model was trained using CLIP embeddings to predict quality scores for new images.

Model Architecture

  • Input: Concatenated CLIP image and text embeddings.
  • Output: Predictions for four quality metrics.

#### Network Structure:

plaintext
Linear(1024, 256) → ReLU
Linear(256, 4) → Sigmoid

Training Setup

  • Optimizer: Adam *(learning rate = 1e-4)*
  • Loss Function: Weighted Binary Cross-Entropy *(compensating for class imbalance)*
  • Epochs: 100 *(early stopping applied)*
  • Batch Size: 512

---

4. Evaluation (Evaluation.ipynb)

The trained model was tested on a held-out test dataset, and its performance was analyzed across multiple metrics.

Model Performance Metrics

The evaluation was conducted using multiple metrics to assess accuracy, precision, recall, and F1-score across different categories.

#### Training Results
| Metric | Overall |
|----------|---------|
| Accuracy | 0.8326 |
| Precision | 0.9930 |
| Recall | 0.9576 |
| F1 Score | 0.9746 |

#### Per-Category Metrics:
| Category | Accuracy | Precision | Recall | F1 Score |
|---------------|-----------|-----------|--------|---------|
| Relevance | 0.9633 | 0.9887 | 0.9712 | 0.9799 |
| Visual Quality | 0.9895 | 0.9990 | 0.9903 | 0.9947 |
| Artifacts | 0.8860 | 0.9895 | 0.8881 | 0.9360 |
| Final Probability | 0.9765 | 0.9948 | 0.9808 | 0.9878 |

#### Validation Results
| Metric | Overall |
|----------|---------|
| Accuracy | 0.7603 |
| Precision | 0.9699 |
| Recall | 0.9466 |
| F1 Score | 0.9579 |

#### Per-Category Metrics:
| Category | Accuracy | Precision | Recall | F1 Score |
|---------------|-----------|-----------|--------|---------|
| Relevance | 0.9151 | 0.9553 | 0.9508 | 0.9530 |
| Visual Quality | 0.9682 | 0.9883 | 0.9790 | 0.9836 |
| Artifacts | 0.8571 | 0.9607 | 0.8840 | 0.9208 |
| Final Probability | 0.9505 | 0.9755 | 0.9726 | 0.9740 |

Test Results

#### Overall Metrics
| Metric | Value |
|-------------|--------|
| Accuracy | 0.7611 |
| Precision | 0.9741 |
| Recall | 0.9480 |
| F1 Score | 0.9606 |

#### Per-Category Metrics
| Category | Accuracy | Precision | Recall | F1 Score |
|---------------|-----------|-----------|--------|---------|
| Relevance | 0.9281 | 0.9652 | 0.9563 | 0.9607 |
| Visual Quality | 0.9767 | 0.9913 | 0.9848 | 0.9881 |
| Artifacts | 0.8478 | 0.9577 | 0.8750 | 0.9145 |
| Final Probability | 0.9598 | 0.9822 | 0.9757 | 0.9790 |

The model corresponding to the reported results can be accessed at:

---

5. Conclusion

The model achieved promising results given the imposed constraints on cost, time and computational resources. Despite these limitations, the evaluation metrics indicate that the approach is effective for assessing AI-generated human images.

Constraints and Future Improvements

  • GPT-4o Mini was chosen over GPT-4o due to its significantly lower cost and faster processing speed. However, GPT-4o is a much more powerful model and could improve results significantly if used.
  • The dataset size was capped at 10,000 images due to computational and financial constraints. A larger dataset would likely enhance performance.
  • A lightweight head network was selected to balance computational feasibility with effective learning. A more complex model could further improve results but requires more computational power.

By addressing these areas, the model can achieve significantly better results in real-world applications.

查看官方来源