ms eff gcvit deepfake b5 celeb df v2

ProviderKoreaPeter
Categoryvideo-classification
Licensemit
Downloads13.4K
Stars0

Overview

The ms-eff-gcvit-deepfake-b5 model is a specialized video classification tool designed to detect synthetic facial manipulations in high-resolution celebrity footage. Built on an efficient GCViT architecture, it optimizes the trade-off between spatial accuracy and computational overhead, making it suitable for production pipelines where latency is a concern. Developers can integrate this model into verification workflows or content moderation systems to flag AI-generated deepfakes. Unlike generic image classifiers, this model leverages temporal consistency across video frames to identify subtle artifacts common in GAN and diffusion-based face-swaps, offering a more robust detection mechanism for dynamic media.

Highlights

  • Optimized GCViT architecture for efficient video frame analysis
  • Specialized in detecting high-fidelity celebrity deepfake manipulations
  • Low-latency inference suitable for real-time content moderation
  • Permissive MIT license for flexible commercial integration

Usage

Install
# Install Hugging Face transformers
pip install transformers torch
SDK Usage
# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("KoreaPeter/ms-eff-gcvit-deepfake-b5-celeb-df-v2")
tokenizer = AutoTokenizer.from_pretrained("KoreaPeter/ms-eff-gcvit-deepfake-b5-celeb-df-v2")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance 
pip install -U huggingface_hub

CLI Download

Download the full repository

Download the full repository 
huggingface-cli download KoreaPeter/ms-eff-gcvit-deepfake-b5-celeb-df-v2

Download a single file to a local folder (e.g. config.json into ./dir)

Download a single file to a local folder (e.g. config.json into ./dir) 
huggingface-cli download KoreaPeter/ms-eff-gcvit-deepfake-b5-celeb-df-v2 config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

SDK Download 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('KoreaPeter/ms-eff-gcvit-deepfake-b5-celeb-df-v2')

Git Download

Make sure git-lfs is installed first

Git Download 
git lfs install
git clone https://huggingface.co/KoreaPeter/ms-eff-gcvit-deepfake-b5-celeb-df-v2

To skip LFS large-file downloads, use:

Skip LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/KoreaPeter/ms-eff-gcvit-deepfake-b5-celeb-df-v2

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

Install Transformers 
pip install -U transformers torch

Load the model and run inference

Load the model and run inference 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('KoreaPeter/ms-eff-gcvit-deepfake-b5-celeb-df-v2')
tokenizer = AutoTokenizer.from_pretrained('KoreaPeter/ms-eff-gcvit-deepfake-b5-celeb-df-v2')

Full Documentation

来源: HuggingFace

---
license: mit
datasets:

  • ILSVRC/imagenet-1k

metrics:
  • accuracy

  • roc_auc

base_model:
  • timm/tf_efficientnet_b5.ns_jft_in1k

pipeline_tag: video-classification
library_name: transformers
tags:
  • PyTorch

  • vision

  • DeepFake-Detection

  • DeepGuard

  • tf-efficientnet

  • global-context-vision-transformer

---

🚀 Multi Scale Efficient Global Context Vision Transformer

!Task
!Image Classification
!Video Classification

!FaceForensics++
!Celeb-DF(v2)-00C853?style=flat-square)
!KODF

<img src="./ms_eff_gcvit.JPG" width="900">

> 🔗 GitHub Repository: HanMoonSub/DeepGuard

> 🤗 Live demo: DeepFake Video Detection

> 🤗 Live demo: DeepFake Image Detection

> 🤗 Live demo: DeepFake Detection XAI

Multi-Scale Efficient Global Context Vision Transformer (MS-EffGCViT) is a hybrid CNN-ViT
architecture for deepfake detection. It fuses CNN-driven spatial inductive bias with
hierarchical global-context attention to catch both *local* artifacts (textures, blending seams)
and *global* artifacts (lighting, structural inconsistency).

A single architecture ships in two sizes and three domain-tuned checkpoints, working on both
static images and video at the frame level.

✨ Core Features

  • 🎞️ Frame-level — one model handles both images and videos (frame-level inference + aggregation).
  • 🌍 Cross-domain — robust on both East-Asian (KoDF) and Western (Celeb-DF-v2, FaceForensics++) faces.
  • ⚡🔥 Two variants — Fast (b0) for real-time/edge, Pro (b5) for enterprise accuracy.
  • 🧩 timm-compatible — load via the timm interface or the deepguard package.

⚙️ Model Specifications

| Spec | Detail | |---|---| | Task | Binary deepfake detection (real / fake) | | Domain | Frame-level, spatial-domain | | Input | Image or video (face-cropped) | | Output | Sigmoid probability in [0, 1] — higher = more likely fake | | Backbone | EfficientNet (ImageNet-1K pretrained) | | Framework | PyTorch / timm |

🧬 Model Zoo

⚡ ms_eff_gcvit_b0 is Optimized for real-time inference and mobile deployment.

🔥 ms_eff_gcvit_b5 is Engineered for high-fidelity analysis and enterprise-grade accuracy.

| Config | ⚡ Fast (b0) | 🔥 Pro (b5) |
|---|---|---|
| Model name | ms_eff_gcvit_b0 | ms_eff_gcvit_b5 |
| Backbone | tf_efficientnet_b0.ns_jft_in1k | tf_efficientnet_b5.ns_jft_in1k |
| Resolution | 224×224 | 384×384 |
| Params (M) | 8.7 | 50.3 |
| FLOPs (G) | 0.87 | 13.64 |

📚 Dataset: Celeb-DF-v2

A large-scale challenging dataset for deepfake forensics [[Paper]](https://openaccess.thecvf.com/content_CVPR_2020/papers/Li_Celeb-DF_A_Large-Scale_Challenging_Dataset_for_DeepFake_Forensics_CVPR_2020_paper.pdf) [[Download]](https://github.com/yuezunli/celeb-deepfakeforensics/tree/master),
featuring 590 YouTube celebrity videos with diverse ages, ethnic groups, and genders.

  • [x] Source: 590 original YouTube videos (celebrities)
  • [x] Synthesis: 5,639 deepfake videos generated from real videos
  • [x] Subjects: Diverse ages, ethnicities, and genders

| Source | Real/Fake | Videos | Description |
| ------ | --------- | ------ | ----------- |
| celeb-real | !Real | 590 | Celebrity videos from YouTube |
| youtube-real | !Real | 300 | Additional YouTube videos |
| celeb-synthesis | !Fake | 5,639 | Synthesized from celeb-real |

> 📎 Available at GitHub or Kaggle

📈 Test Evaluation

Trained and tested on the same dataset.

<img src="./celeb_df_v2_gcvit.png" width="900">

| Dataset | Variant | Accuracy | AUC | Log Loss |
|---|---|---|---|---|
| Celeb-DF-v2 | ⚡ Fast | 0.9842 | 0.9965 | 0.0283 |
| Celeb-DF-v2 | 🔥 Pro | 0.9981 | 0.9984 | 0.0089 |

📈 Cross-Dataset Evaluation (Trained on Celeb DF(v2))

Generalization to unseen domains — trained on Celeb DF(v2)

| Tested on | Variant | Accuracy | AUC | Log Loss |
|---|---|---|---|---|
| KoDF | ⚡ Fast | 0.4935 | 0.7258 | 1.3459 |
| KoDF | 🔥 Pro | 0.4832 | 0.7160 | 1.4897 |
| FaceForensics++ | ⚡ Fast | 0.5492 | 0.7301 | 1.0556 |
| FaceForensics++ | 🔥 Pro | 0.5825 | 0.7307 | 0.8897 |


🚀 Model Usage

python
pip install deepguard
from transformers import pipeline

🖼️ Image Classification

python
clf = pipeline(
    "image-classification",
    model="KoreaPeter/ms-eff-gcvit-deepfake-b5-celeb-df-v2", 
    trust_remote_code=True,
)

── Basic Inference ───────────────────────────────────────────────

result = clf("face.jpg")

[{'label': 'fake', 'score': 0.9712}, {'label': 'real', 'score': 0.0288}]


── Custom Parameters ─────────────────────────────────────────────

result = clf(
    "face.jpg",
    margin_ratio=0.2,      # Margin ratio around the detected face bbox (default: 0.2)
    conf_thres=0.5,        # Confidence threshold for YOLO face detection (default: 0.5)
    min_face_ratio=0.01,   # Minimum face-to-frame area ratio to process (default: 0.01)
    tta_hflip=0.0,         # Probability of horizontal flip for TTA (default: 0.0)
    top_k=1,               # Number of top labels to return (default: all)
)

[{'label': 'fake', 'score': 0.9712}]

🎬 Video Classification

python
clf = pipeline(
    "video-classification",
    model="KoreaPeter/ms-eff-gcvit-deepfake-b5-celeb-df-v2",  
    trust_remote_code=True,
)

── Basic Inference ───────────────────────────────────────────────

result = clf("video.mp4")

[{'label': 'fake', 'score': 0.9634}, {'label': 'real', 'score': 0.0366}]


── Custom Parameters ─────────────────────────────────────────────

result = clf(
    "video.mp4",
    num_frames=20,              # Number of frames to sample (default: 20)
    margin_ratio=0.2,           # Margin ratio around the detected face bbox (default: 0.2)
    conf_thres=0.5,             # Confidence threshold for YOLO face detection (default: 0.5)
    min_face_ratio=0.01,        # Minimum face-to-frame area ratio to process (default: 0.01)
    tta_hflip=0.0,              # Probability of horizontal flip for TTA (default: 0.0)
    agg_mode="conf",            # Aggregation mode: 'conf' | 'mean' | 'vote' (default: 'conf')
    return_frame_scores=True,   # Return per-frame scores (default: False)
)

[{'label': 'fake', 'score': 0.9634},


 {'label': 'real', 'score': 0.0366},


 {'frame_scores': [0.97, 0.95, 0.98, ...], 'agg_mode': 'conf'}]

Deep Dive into Model

Part 1: CNN-based Patch Embedding for Spatial Inductive Bias

While traditional Vision Transformers (ViTs) utilize a Linear Projection for patch embedding, our proposed model adopts a CNN-based Patch Embedding module incorporating MBConvBlocks.

  • Injecting Inductive Bias : Standard ViTs often suffer from a lack of inherent spatial inductive bias, typically necessitating massive datasets to learn fundamental visual structures from scratch. In contrast, our CNN-based module leverages overlapping receptive fields to facilitate information sharing between neighboring patches.
View official source
Join our Telegram