Overview

ModNet is a lightweight, real-time portrait matting model designed for high-fidelity alpha matte estimation. Unlike traditional segmentation models that produce binary masks, ModNet focuses on soft edges, making it ideal for tasks like virtual background replacement and portrait blurring where hair and fine details are critical. It is optimized for efficiency, allowing for deployment in browser-based environments or edge devices without requiring heavy GPU overhead. For developers, this means a streamlined integration path for image processing pipelines that need to isolate subjects from complex backgrounds with professional-grade transparency.

Highlights

Real-time portrait matting with high-quality alpha matte output
Efficient architecture optimized for edge and browser deployment
Superior handling of fine details like hair and edges
Permissive Apache-2.0 license for commercial application
Seamless integration into image processing and AR pipelines

Usage

Install

# Install Hugging Face transformers
pip install transformers torch

SDK Usage

# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("Xenova/modnet")
tokenizer = AutoTokenizer.from_pretrained("Xenova/modnet")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance

pip install -U huggingface_hub

CLI Download

Download the full repository

huggingface-cli download Xenova/modnet

Download a single file to a local folder (e.g. config.json into ./dir)

huggingface-cli download Xenova/modnet config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('Xenova/modnet')

Git Download

Make sure git-lfs is installed first

Git Download

git lfs install
git clone https://huggingface.co/Xenova/modnet

To skip LFS large-file downloads, use:

Skip LFS

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Xenova/modnet

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

pip install -U transformers torch

Load the model and run inference

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('Xenova/modnet')
tokenizer = AutoTokenizer.from_pretrained('Xenova/modnet')

Model Download

We recommend downloading the model via the ModelScope CLI or SDK.

Guidance：Before downloading, install ModelScope with:

Guidance

pip install modelscope

CLI Download

Download the full repository

modelscope download --model Xenova/modnet

Download a single file to a local folder (e.g. README.md into ./dir)

modelscope download --model Xenova/modnet README.md --local_dir ./dir

See the docs for more CLI options

SDK Download

# 模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('Xenova/modnet')

Git Download

Make sure git-lfs is installed first

Git Download

git lfs install
git clone https://www.modelscope.cn/Xenova/modnet.git

To skip LFS large-file downloads, use:

Skip LFS

GIT_LFS_SKIP_SMUDGE=1 git clone https://www.modelscope.cn/Xenova/modnet.git

ModelScope 模型页直接下载模型文件；无需将模型文件放在本站服务器。

Notebook Quickstart

Install the ModelScope library

pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html

Load the model and run inference

from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

p = pipeline('text-generation', 'Xenova/modnet')

Full Documentation

来源: HuggingFace

---
library_name: transformers.js
tags:

vision

background-removal

portrait-matting

license: apache-2.0
pipeline_tag: image-segmentation
---

MODNet: Trimap-Free Portrait Matting in Real Time

!image/gif

For more information, check out the official repository and example colab.

Usage (Transformers.js)

If you haven't already, you can install the Transformers.js JavaScript library from NPM using:

bash

npm i @huggingface/transformers

You can then use the model for portrait matting, as follows:

js

import { pipeline } from '@huggingface/transformers';
const segmenter = await pipeline('background-removal', 'Xenova/modnet', { dtype: 'fp32' });
const url = 'https://images.pexels.com/photos/5965592/pexels-photo-5965592.jpeg?auto=compress&cs=tinysrgb&w=1024';
const output = await segmenter(url);
output[0].save('mask.png');
// You can also use output[0].toCanvas() or await output[0].toBlob() if you would like to access the output without saving.

Or with the AutoModel and AutoProcessor APIs:

js

import { AutoModel, AutoProcessor, RawImage } from '@huggingface/transformers';
// Load model and processor
const model = await AutoModel.from_pretrained('Xenova/modnet', { dtype: 'fp32' });
const processor = await AutoProcessor.from_pretrained('Xenova/modnet');
// Load image from URL
const url = 'https://images.pexels.com/photos/5965592/pexels-photo-5965592.jpeg?auto=compress&cs=tinysrgb&w=1024';
const image = await RawImage.fromURL(url);
// Pre-process image
const { pixel_values } = await processor(image);
// Predict alpha matte
const { output } = await model({ input: pixel_values });
// Save output mask
const mask = await RawImage.fromTensor(output[0].mul(255).to('uint8')).resize(image.width, image.height);
mask.save('mask.png');

| Input image | Output mask |
|--------|--------|
| !image/png | !image/png |

---

Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using 🤗 Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).