Overview

The coco panoptic eomt large 640 is a specialized image segmentation model designed for high-precision panoptic tasks. Unlike standard semantic segmentation, this model simultaneously handles instance segmentation (identifying individual objects) and semantic segmentation (classifying background textures), operating at a 640px resolution. For developers, this means a unified pipeline for scene understanding without needing separate models for 'things' and 'stuff'. It is particularly effective for robotics, autonomous navigation, and visual analysis where spatial boundaries and object counts are critical. With an MIT license, it offers significant flexibility for commercial integration into existing CV pipelines via standard PyTorch or ONNX runtimes.

Highlights

Unified panoptic segmentation for objects and backgrounds
Optimized for 640px input resolution
Permissive MIT license for commercial deployment
High-precision boundary detection for complex scenes

Usage

Install

# Install Hugging Face transformers
pip install transformers torch

SDK Usage

# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("tue-mps/coco_panoptic_eomt_large_640")
tokenizer = AutoTokenizer.from_pretrained("tue-mps/coco_panoptic_eomt_large_640")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance

pip install -U huggingface_hub

CLI Download

Download the full repository

huggingface-cli download tue-mps/coco_panoptic_eomt_large_640

Download a single file to a local folder (e.g. config.json into ./dir)

huggingface-cli download tue-mps/coco_panoptic_eomt_large_640 config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('tue-mps/coco_panoptic_eomt_large_640')

Git Download

Make sure git-lfs is installed first

Git Download

git lfs install
git clone https://huggingface.co/tue-mps/coco_panoptic_eomt_large_640

To skip LFS large-file downloads, use:

Skip LFS

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/tue-mps/coco_panoptic_eomt_large_640

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

pip install -U transformers torch

Load the model and run inference

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('tue-mps/coco_panoptic_eomt_large_640')
tokenizer = AutoTokenizer.from_pretrained('tue-mps/coco_panoptic_eomt_large_640')

Full Documentation

来源: HuggingFace

---
library_name: transformers
license: mit
tags:

vision

image-segmentation

pytorch

---

EoMT

![PyTorch](https://pytorch.org/)

EoMT (Encoder-only Mask Transformer) is a Vision Transformer (ViT) architecture designed for high-quality and efficient image segmentation. It was introduced in the CVPR 2025 highlight paper:
Your ViT is Secretly an Image Segmentation Model
by Tommie Kerssies, Niccolò Cavagnero, Alexander Hermans, Narges Norouzi, Giuseppe Averta, Bastian Leibe, Gijs Dubbelman, and Daan de Geus.

> Key Insight: Given sufficient scale and pretraining, a plain ViT along with additional few params can perform segmentation without the need for task-specific decoders or pixel fusion modules. The same model backbone supports semantic, instance, and panoptic segmentation with different post-processing 🤗

The original implementation can be found in this repository.

The HuggingFace model page is available at this link.

---

How to use

Here is how to use this model for Panotpic Segmentation:

python

import matplotlib.pyplot as plt
import requests
import torch
from PIL import Image
from transformers import EomtForUniversalSegmentation, AutoImageProcessor
model_id = "tue-mps/coco_panoptic_eomt_large_640"
processor = AutoImageProcessor.from_pretrained(model_id)
model = EomtForUniversalSegmentation.from_pretrained(model_id)
image = Image.open(requests.get("http://images.cocodataset.org/val2017/000000039769.jpg", stream=True).raw)
inputs = processor(
    images=image,
    return_tensors="pt",
)
with torch.inference_mode():
    outputs = model(**inputs)
Prepare the original image size in the format (height, width)
target_sizes = [(image.height, image.width)]
Post-process the model outputs to get final segmentation prediction
preds = processor.post_process_panoptic_segmentation(
    outputs,
    target_sizes=target_sizes,
)
Visualize the panoptic segmentation mask
plt.imshow(preds[0]["segmentation"])
plt.axis("off")
plt.title("Panoptic Segmentation")
plt.show()

Citation

If you find our work useful, please consider citing us as:

bibtex

@inproceedings{kerssies2025eomt,
  author    = {Kerssies, Tommie and Cavagnero, Niccolò and Hermans, Alexander and Norouzi, Narges and Averta, Giuseppe and Leibe, Bastian and Dubbelman, Gijs and de Geus, Daan},
  title     = {Your ViT is Secretly an Image Segmentation Model},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2025},
}