ms eff gcvit deepfake b5 kodf

---
license: mit
datasets:

• ILSVRC/imagenet-1k

• accuracy

• roc_auc

• timm/tf_efficientnet_b5.ns_jft_in1k

• PyTorch

• vision

• DeepFake-Detection

• DeepGuard

• tf-efficientnet

• global-context-vision-transformer

🚀 Multi Scale Efficient Global Context Vision Transformer

!Task
!Image Classification
!Video Classification

!FaceForensics++
!Celeb-DF(v2)-00C853?style=flat-square)
!KODF

<img src="./ms_eff_gcvit.JPG" width="900">

> 🔗 GitHub Repository: HanMoonSub/DeepGuard

> 🤗 Live demo: DeepFake Video Detection

> 🤗 Live demo: DeepFake Image Detection

> 🤗 Live demo: DeepFake Detection XAI

Multi-Scale Efficient Global Context Vision Transformer (MS-EffGCViT) is a hybrid CNN-ViT
architecture for deepfake detection. It fuses CNN-driven spatial inductive bias with
hierarchical global-context attention to catch both *local* artifacts (textures, blending seams)
and *global* artifacts (lighting, structural inconsistency).

A single architecture ships in two sizes and three domain-tuned checkpoints, working on both
static images and video at the frame level.

✨ Core Features

• 🎞️ Frame-level — one model handles both images and videos (frame-level inference + aggregation).

• 🌍 Cross-domain — robust on both East-Asian (KoDF) and Western (Celeb-DF-v2, FaceForensics++) faces.

• ⚡🔥 Two variants — Fast (b0) for real-time/edge, Pro (b5) for enterprise accuracy.

• 🧩 timm-compatible — load via the timm interface or the deepguard package.

⚙️ Model Specifications

🧬 Model Zoo

⚡ ms_eff_gcvit_b0 is Optimized for real-time inference and mobile deployment.

🔥 ms_eff_gcvit_b5 is Engineered for high-fidelity analysis and enterprise-grade accuracy.

| Config | ⚡ Fast (b0) | 🔥 Pro (b5) |
|---|---|---|
| Model name | ms_eff_gcvit_b0 | ms_eff_gcvit_b5 |
| Backbone | tf_efficientnet_b0.ns_jft_in1k | tf_efficientnet_b5.ns_jft_in1k |
| Resolution | 224×224 | 384×384 |
| Params (M) | 8.7 | 50.3 |
| FLOPs (G) | 0.87 | 13.64 |

📚 Dataset: KoDF (Korean Deepfake Dataset)

Large-scale Korean deepfake dataset provided by AI-Hub in 2020

featuring 400 Korean participants across 6 deepfake synthesis methods

• [x] Number of Subjects: 400 participants

• [x] Videos per Subject: 150+ videos

• [x] Total Data Duration: 88.5 days

• [x] Deepfake Model Variants: 6 types(DeepFaceLab, FaceSwap, FSGAN, FOMM, 3DMM, Wav2Lip)

| Metric | Original Data | Fake Data |
| ------ | ------------- | --------- |
| Total Videos | 62,166 | 175,776 |
| Average Video Length | 90+ second | 15+ second |
| Total Duration | 1,500+ hours | 625+ hours|
| Resolution | 1920 X 1080 | 1920 X 1080 |
| FPS | 30 FPS | 30 FPS |
| Total Frames | 162,000,000+ | -- |

📈 Test Evaluation

Trained and tested on the same dataset.

| Dataset | Variant | Accuracy | AUC | Log Loss |
|---|---|---|---|---|
| KoDF | ⚡ Fast | 0.9655 | 0.9792 | 0.1237 |
| KoDF | 🔥 Pro | 0.9792 | 0.9831 | 0.0692 |

📈 Cross-Dataset Evaluation (Trained on KoDF)

Generalization to unseen domains — trained on KoDF, evaluated on western-face datasets.

| Tested on | Variant | Accuracy | AUC | Log Loss |
|---|---|---|---|---|
| Celeb-DF-v2 | ⚡ Fast | 0.5579 | 0.4719 | 1.2605 |
| Celeb-DF-v2 | 🔥 Pro | 0.5946 | 0.5400 | 1.0078 |
| FaceForensics++ | ⚡ Fast | 0.4875 | 0.5341 | 1.6178 |
| FaceForensics++ | 🔥 Pro | 0.4525 | 0.5902 | 1.5321 |

🚀 Model Usage

pip install deepguard
from transformers import pipeline

🖼️ Image Classification

clf = pipeline(
    "image-classification",
    model="KoreaPeter/ms-eff-gcvit-deepfake-b5-kodf", 
    trust_remote_code=True,
)
── Basic Inference ───────────────────────────────────────────────
result = clf("face.jpg")
[{'label': 'fake', 'score': 0.9712}, {'label': 'real', 'score': 0.0288}]
── Custom Parameters ─────────────────────────────────────────────
result = clf(
    "face.jpg",
    margin_ratio=0.2,      # Margin ratio around the detected face bbox (default: 0.2)
    conf_thres=0.5,        # Confidence threshold for YOLO face detection (default: 0.5)
    min_face_ratio=0.01,   # Minimum face-to-frame area ratio to process (default: 0.01)
    tta_hflip=0.0,         # Probability of horizontal flip for TTA (default: 0.0)
    top_k=1,               # Number of top labels to return (default: all)
)
[{'label': 'fake', 'score': 0.9712}]

🎬 Video Classification

clf = pipeline(
    "video-classification",
    model="KoreaPeter/ms-eff-gcvit-deepfake-b5-kodf",
    trust_remote_code=True,
)
── Basic Inference ───────────────────────────────────────────────
result = clf("video.mp4")
[{'label': 'fake', 'score': 0.9634}, {'label': 'real', 'score': 0.0366}]
── Custom Parameters ─────────────────────────────────────────────
result = clf(
    "video.mp4",
    num_frames=20,              # Number of frames to sample (default: 20)
    margin_ratio=0.2,           # Margin ratio around the detected face bbox (default: 0.2)
    conf_thres=0.5,             # Confidence threshold for YOLO face detection (default: 0.5)
    min_face_ratio=0.01,        # Minimum face-to-frame area ratio to process (default: 0.01)
    tta_hflip=0.0,              # Probability of horizontal flip for TTA (default: 0.0)
    agg_mode="conf",            # Aggregation mode: 'conf' | 'mean' | 'vote' (default: 'conf')
    return_frame_scores=True,   # Return per-frame scores (default: False)
)
[{'label': 'fake', 'score': 0.9634},
 {'label': 'real', 'score': 0.0366},
 {'frame_scores': [0.97, 0.95, 0.98, ...], 'agg_mode': 'conf'}]

Deep Dive into Model

Part 1: CNN-based Patch Embedding for Spatial Inductive Bias

While traditional Vision Transformers (ViTs) utilize a Linear Projection for patch embedding, our proposed model adopts a CNN-based Patch Embedding module incorporating MBConvBlocks.

• Injecting Inductive Bias : Standard ViTs often suffer from a lack of inherent spatial inductive bias, typically necessitating massive datasets to learn fundamental visual structures from scratch. In contrast, our CNN-based module leverages overlapping receptive fields to facilitate information sharing between neighboring patches. By explicitly injecting this spatial bias into the architecture, the model achieves more stable and accelerated convergence during the training process.

Part 2: Long-Short Range Spatial Interaction

We utilizes two distinct types of self-attention to capture both long-range and short-range information across feature maps.

<img src="./window_attention.JPG" width="900">

• Local Window Attention: this model efficiently captures local textures and precise spatial details while maintaining linear c