BiRefNet

Name: BiRefNet
Author: ZhengPeng7

提供商ZhengPeng7

分类image-segmentation

许可证mit

下载量804.0K

星标0

简介

BiRefNet 是一款专注于高质量图像背景去除（抠图）的深度学习模型。与传统的分割模型不同，它在处理物体边缘（如发丝、半透明材质）时表现极其细腻，能够有效避免粗糙的锯齿感。对于开发者而言，该模型上手门槛低，非常适合集成到电商自动抠图、证件照处理或创意设计工具中。如果你觉得目前常用的 AI 抠图工具在细节上不够精准，BiRefNet 提供了一个高性能的开源替代方案，能够显著提升图像素材的纯净度。

核心亮点

极致边缘处理，发丝级抠图精度
开源 MIT 协议，商业集成无压力
无需复杂调参，即插即用处理背景
理想的电商与设计类自动化预处理工具

使用方法

安装依赖

# 安装 Hugging Face transformers
pip install transformers torch

SDK 使用

# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("ZhengPeng7/BiRefNet")
tokenizer = AutoTokenizer.from_pretrained("ZhengPeng7/BiRefNet")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引

pip install -U huggingface_hub

命令行下载

下载完整模型库

huggingface-cli download ZhengPeng7/BiRefNet

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

huggingface-cli download ZhengPeng7/BiRefNet config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('ZhengPeng7/BiRefNet')

Git 下载

请确保 lfs 已经被正确安装

Git 下载

git lfs install
git clone https://huggingface.co/ZhengPeng7/BiRefNet

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/ZhengPeng7/BiRefNet

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

pip install -U transformers torch

模型加载和推理

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('ZhengPeng7/BiRefNet')
tokenizer = AutoTokenizer.from_pretrained('ZhengPeng7/BiRefNet')

完整文档

来源: HuggingFace

---
library_name: birefnet
tags:

background-removal

mask-generation

Dichotomous Image Segmentation

Camouflaged Object Detection

Salient Object Detection

pytorch_model_hub_mixin

model_hub_mixin

transformers

repo_url: https://github.com/ZhengPeng7/BiRefNet
pipeline_tag: image-segmentation
license: mit
---
<h1 align="center">Bilateral Reference for High-Resolution Dichotomous Image Segmentation</h1>

<div align='center'>
<a href='https://scholar.google.com/citations?user=TZRzWOsAAAAJ' target='_blank'>Peng Zheng</a> 1,4,5,6, 
<a href='https://scholar.google.com/citations?user=0uPb8MMAAAAJ' target='_blank'>Dehong Gao</a> 2, 
<a href='https://scholar.google.com/citations?user=kakwJ5QAAAAJ' target='_blank'>Deng-Ping Fan</a> 1*, 
<a href='https://scholar.google.com/citations?user=9cMQrVsAAAAJ' target='_blank'>Li Liu</a> 3, 
<a href='https://scholar.google.com/citations?user=qQP6WXIAAAAJ' target='_blank'>Jorma Laaksonen</a> 4, 
<a href='https://scholar.google.com/citations?user=pw_0Z_UAAAAJ' target='_blank'>Wanli Ouyang</a> 5, 
<a href='https://scholar.google.com/citations?user=stFCYOAAAAAJ' target='_blank'>Nicu Sebe</a> 6
</div>

<div align='center'>
1 Nankai University&ensp; 2 Northwestern Polytechnical University&ensp; 3 National University of Defense Technology&ensp; 4 Aalto University&ensp; 5 Shanghai AI Laboratory&ensp; 6 University of Trento&ensp;
</div>

| *DIS-Sample_1* | *DIS-Sample_2* |
| :------------------------------: | :-------------------------------: |
| <img src="https://drive.google.com/thumbnail?id=1ItXaA26iYnE8XQ_GgNLy71MOWePoS2-g&sz=w400" /> | <img src="https://drive.google.com/thumbnail?id=1Z-esCujQF_uEa_YJjkibc3NUrW4aR_d4&sz=w400" /> |

This repo is the official implementation of "Bilateral Reference for High-Resolution Dichotomous Image Segmentation" (___CAAI AIR 2024___).

Visit our GitHub repo: https://github.com/ZhengPeng7/BiRefNet for more details -- codes, docs, and model zoo!

How to use

0. Install Packages:

code

pip install -qr https://raw.githubusercontent.com/ZhengPeng7/BiRefNet/main/requirements.txt

1. Load BiRefNet:

#### Use codes + weights from HuggingFace
> Only use the weights on HuggingFace -- Pro: No need to download BiRefNet codes manually; Con: Codes on HuggingFace might not be latest version (I'll try to keep them always latest).

python

# Load BiRefNet with weights
from transformers import AutoModelForImageSegmentation
birefnet = AutoModelForImageSegmentation.from_pretrained('ZhengPeng7/BiRefNet', trust_remote_code=True)

#### Use codes from GitHub + weights from HuggingFace
> Only use the weights on HuggingFace -- Pro: codes are always latest; Con: Need to clone the BiRefNet repo from my GitHub.

shell

# Download codes
git clone https://github.com/ZhengPeng7/BiRefNet.git
cd BiRefNet

python

# Use codes locally
from models.birefnet import BiRefNet
Load weights from Hugging Face Models
birefnet = BiRefNet.from_pretrained('ZhengPeng7/BiRefNet')

#### Use codes from GitHub + weights from local space
> Only use the weights and codes both locally.

python

# Use codes and weights locally
import torch
from utils import check_state_dict
birefnet = BiRefNet(bb_pretrained=False)
state_dict = torch.load(PATH_TO_WEIGHT, map_location='cpu')
state_dict = check_state_dict(state_dict)
birefnet.load_state_dict(state_dict)

#### Use the loaded BiRefNet for inference

python

# Imports
from PIL import Image
import matplotlib.pyplot as plt
import torch
from torchvision import transforms
from models.birefnet import BiRefNet
birefnet = ... # -- BiRefNet should be loaded with codes above, either way.
torch.set_float32_matmul_precision(['high', 'highest'][0])
birefnet.to('cuda')
birefnet.eval()
birefnet.half()
def extract_object(birefnet, imagepath):
    # Data settings
    image_size = (1024, 1024)
    transform_image = transforms.Compose([
        transforms.Resize(image_size),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])
image = Image.open(imagepath)
    input_images = transform_image(image).unsqueeze(0).to('cuda').half()
# Prediction
    with torch.no_grad():
        preds = birefnet(input_images)[-1].sigmoid().cpu()
    pred = preds[0].squeeze()
    pred_pil = transforms.ToPILImage()(pred)
    mask = pred_pil.resize(image.size)
    image.putalpha(mask)
    return image, mask
Visualization
plt.axis("off")
plt.imshow(extract_object(birefnet, imagepath='PATH-TO-YOUR_IMAGE.jpg')[0])
plt.show()

2. Use inference endpoint locally:

> You may need to click the *deploy* and set up the endpoint by yourself, which would make some costs.

code

import requests
import base64
from io import BytesIO
from PIL import Image
YOUR_HF_TOKEN = 'xxx'
API_URL = "xxx"
headers = {
	"Authorization": "Bearer {}".format(YOUR_HF_TOKEN)
}
def base64_to_bytes(base64_string):
    # Remove the data URI prefix if present
    if "data:image" in base64_string:
        base64_string = base64_string.split(",")[1]
# Decode the Base64 string into bytes
    image_bytes = base64.b64decode(base64_string)
    return image_bytes
def bytes_to_base64(image_bytes):
    # Create a BytesIO object to handle the image data
    image_stream = BytesIO(image_bytes)
# Open the image using Pillow (PIL)
    image = Image.open(image_stream)
    return image
def query(payload):
	response = requests.post(API_URL, headers=headers, json=payload)
	return response.json()
output = query({
	"inputs": "https://hips.hearstapps.com/hmg-prod/images/gettyimages-1229892983-square.jpg",
	"parameters": {}
})
output_image = bytes_to_base64(base64_to_bytes(output))
output_image

> This BiRefNet for standard dichotomous image segmentation (DIS) is trained on DIS-TR and validated on DIS-TEs and DIS-VD.

This repo holds the offici

查看官方来源

BiRefNet

简介

核心亮点

使用方法

Hugging Face 下载

命令行下载

SDK 下载

Git 下载

PyTorch / Transformers 使用

完整文档

How to use

0. Install Packages:

1. Load BiRefNet:

Load weights from Hugging Face Models

Visualization

2. Use inference endpoint locally:

This repo holds the offici

扫码进交流群