BiRefNet

Name: BiRefNet
Author: ZhengPeng7

ProviderZhengPeng7

Categoryimage-segmentation

Licensemit

Downloads804.0K

Stars0

Overview

BiRefNet is a high-performance image segmentation model specifically engineered for precise binary reference-based background removal. Unlike general-purpose segmentation tools, BiRefNet focuses on delivering clean, high-resolution masks with sharp edge definition, making it ideal for e-commerce product photography, portrait isolation, and automated asset generation. For developers, the model offers a streamlined integration path for pipelines requiring consistent alpha matting without the overhead of complex manual tuning. It competes effectively with proprietary background removal APIs by providing a robust, open-source alternative that maintains high fidelity even with complex boundaries like hair or translucent objects.

Highlights

High-precision binary masks for professional background removal
Superior edge fidelity on complex object boundaries
MIT licensed for flexible commercial and private integration
Optimized for high-resolution image segmentation workflows

Usage

Install

# Install Hugging Face transformers
pip install transformers torch

SDK Usage

# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("ZhengPeng7/BiRefNet")
tokenizer = AutoTokenizer.from_pretrained("ZhengPeng7/BiRefNet")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance

pip install -U huggingface_hub

CLI Download

Download the full repository

huggingface-cli download ZhengPeng7/BiRefNet

Download a single file to a local folder (e.g. config.json into ./dir)

huggingface-cli download ZhengPeng7/BiRefNet config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('ZhengPeng7/BiRefNet')

Git Download

Make sure git-lfs is installed first

Git Download

git lfs install
git clone https://huggingface.co/ZhengPeng7/BiRefNet

To skip LFS large-file downloads, use:

Skip LFS

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/ZhengPeng7/BiRefNet

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

pip install -U transformers torch

Load the model and run inference

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('ZhengPeng7/BiRefNet')
tokenizer = AutoTokenizer.from_pretrained('ZhengPeng7/BiRefNet')

Full Documentation

来源: HuggingFace

---
library_name: birefnet
tags:

background-removal

mask-generation

Dichotomous Image Segmentation

Camouflaged Object Detection

Salient Object Detection

pytorch_model_hub_mixin

model_hub_mixin

transformers

repo_url: https://github.com/ZhengPeng7/BiRefNet
pipeline_tag: image-segmentation
license: mit
---
<h1 align="center">Bilateral Reference for High-Resolution Dichotomous Image Segmentation</h1>

<div align='center'>
<a href='https://scholar.google.com/citations?user=TZRzWOsAAAAJ' target='_blank'>Peng Zheng</a> 1,4,5,6, 
<a href='https://scholar.google.com/citations?user=0uPb8MMAAAAJ' target='_blank'>Dehong Gao</a> 2, 
<a href='https://scholar.google.com/citations?user=kakwJ5QAAAAJ' target='_blank'>Deng-Ping Fan</a> 1*, 
<a href='https://scholar.google.com/citations?user=9cMQrVsAAAAJ' target='_blank'>Li Liu</a> 3, 
<a href='https://scholar.google.com/citations?user=qQP6WXIAAAAJ' target='_blank'>Jorma Laaksonen</a> 4, 
<a href='https://scholar.google.com/citations?user=pw_0Z_UAAAAJ' target='_blank'>Wanli Ouyang</a> 5, 
<a href='https://scholar.google.com/citations?user=stFCYOAAAAAJ' target='_blank'>Nicu Sebe</a> 6
</div>

<div align='center'>
1 Nankai University&ensp; 2 Northwestern Polytechnical University&ensp; 3 National University of Defense Technology&ensp; 4 Aalto University&ensp; 5 Shanghai AI Laboratory&ensp; 6 University of Trento&ensp;
</div>

| *DIS-Sample_1* | *DIS-Sample_2* |
| :------------------------------: | :-------------------------------: |
| <img src="https://drive.google.com/thumbnail?id=1ItXaA26iYnE8XQ_GgNLy71MOWePoS2-g&sz=w400" /> | <img src="https://drive.google.com/thumbnail?id=1Z-esCujQF_uEa_YJjkibc3NUrW4aR_d4&sz=w400" /> |

This repo is the official implementation of "Bilateral Reference for High-Resolution Dichotomous Image Segmentation" (___CAAI AIR 2024___).

Visit our GitHub repo: https://github.com/ZhengPeng7/BiRefNet for more details -- codes, docs, and model zoo!

How to use

0. Install Packages:

code

pip install -qr https://raw.githubusercontent.com/ZhengPeng7/BiRefNet/main/requirements.txt

1. Load BiRefNet:

#### Use codes + weights from HuggingFace
> Only use the weights on HuggingFace -- Pro: No need to download BiRefNet codes manually; Con: Codes on HuggingFace might not be latest version (I'll try to keep them always latest).

python

# Load BiRefNet with weights
from transformers import AutoModelForImageSegmentation
birefnet = AutoModelForImageSegmentation.from_pretrained('ZhengPeng7/BiRefNet', trust_remote_code=True)

#### Use codes from GitHub + weights from HuggingFace
> Only use the weights on HuggingFace -- Pro: codes are always latest; Con: Need to clone the BiRefNet repo from my GitHub.

shell

# Download codes
git clone https://github.com/ZhengPeng7/BiRefNet.git
cd BiRefNet

python

# Use codes locally
from models.birefnet import BiRefNet
Load weights from Hugging Face Models
birefnet = BiRefNet.from_pretrained('ZhengPeng7/BiRefNet')

#### Use codes from GitHub + weights from local space
> Only use the weights and codes both locally.

python

# Use codes and weights locally
import torch
from utils import check_state_dict
birefnet = BiRefNet(bb_pretrained=False)
state_dict = torch.load(PATH_TO_WEIGHT, map_location='cpu')
state_dict = check_state_dict(state_dict)
birefnet.load_state_dict(state_dict)

#### Use the loaded BiRefNet for inference

python

# Imports
from PIL import Image
import matplotlib.pyplot as plt
import torch
from torchvision import transforms
from models.birefnet import BiRefNet
birefnet = ... # -- BiRefNet should be loaded with codes above, either way.
torch.set_float32_matmul_precision(['high', 'highest'][0])
birefnet.to('cuda')
birefnet.eval()
birefnet.half()
def extract_object(birefnet, imagepath):
    # Data settings
    image_size = (1024, 1024)
    transform_image = transforms.Compose([
        transforms.Resize(image_size),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])
image = Image.open(imagepath)
    input_images = transform_image(image).unsqueeze(0).to('cuda').half()
# Prediction
    with torch.no_grad():
        preds = birefnet(input_images)[-1].sigmoid().cpu()
    pred = preds[0].squeeze()
    pred_pil = transforms.ToPILImage()(pred)
    mask = pred_pil.resize(image.size)
    image.putalpha(mask)
    return image, mask
Visualization
plt.axis("off")
plt.imshow(extract_object(birefnet, imagepath='PATH-TO-YOUR_IMAGE.jpg')[0])
plt.show()

2. Use inference endpoint locally:

> You may need to click the *deploy* and set up the endpoint by yourself, which would make some costs.

code

import requests
import base64
from io import BytesIO
from PIL import Image
YOUR_HF_TOKEN = 'xxx'
API_URL = "xxx"
headers = {
	"Authorization": "Bearer {}".format(YOUR_HF_TOKEN)
}
def base64_to_bytes(base64_string):
    # Remove the data URI prefix if present
    if "data:image" in base64_string:
        base64_string = base64_string.split(",")[1]
# Decode the Base64 string into bytes
    image_bytes = base64.b64decode(base64_string)
    return image_bytes
def bytes_to_base64(image_bytes):
    # Create a BytesIO object to handle the image data
    image_stream = BytesIO(image_bytes)
# Open the image using Pillow (PIL)
    image = Image.open(image_stream)
    return image
def query(payload):
	response = requests.post(API_URL, headers=headers, json=payload)
	return response.json()
output = query({
	"inputs": "https://hips.hearstapps.com/hmg-prod/images/gettyimages-1229892983-square.jpg",
	"parameters": {}
})
output_image = bytes_to_base64(base64_to_bytes(output))
output_image

> This BiRefNet for standard dichotomous image segmentation (DIS) is trained on DIS-TR and validated on DIS-TEs and DIS-VD.

This repo holds the offici

View official source