vjepa2 vitg fpc64 256

Providerfacebook
Categoryvideo-classification
Licenseapache-2.0
Downloads398
Stars0

Overview

V-JEPA (ViT-G FPC64 256) is a sophisticated video representation model designed for self-supervised learning. Unlike traditional generative models, it employs a Joint-Embedding Predictive Architecture to learn spatial-temporal features by predicting missing parts of a video in a latent space. For developers, this means the model excels at understanding complex motion and object interactions without the computational overhead of pixel-level reconstruction. It is particularly effective as a frozen backbone for downstream video classification, action recognition, and temporal analysis tasks. Integrating this model allows for high-efficiency transfer learning, as it provides rich, semantic embeddings that significantly reduce the amount of labeled data required to train specialized video classifiers.

Highlights

  • Self-supervised latent space prediction for efficient video understanding
  • Optimized ViT-G architecture for high-capacity temporal feature extraction
  • Ideal backbone for action recognition and video classification
  • Apache-2.0 license ensures flexible commercial and research integration

Usage

Install
# Install Hugging Face transformers
pip install transformers torch
SDK Usage
# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("facebook/vjepa2-vitg-fpc64-256")
tokenizer = AutoTokenizer.from_pretrained("facebook/vjepa2-vitg-fpc64-256")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance 
pip install -U huggingface_hub

CLI Download

Download the full repository

Download the full repository 
huggingface-cli download facebook/vjepa2-vitg-fpc64-256

Download a single file to a local folder (e.g. config.json into ./dir)

Download a single file to a local folder (e.g. config.json into ./dir) 
huggingface-cli download facebook/vjepa2-vitg-fpc64-256 config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

SDK Download 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('facebook/vjepa2-vitg-fpc64-256')

Git Download

Make sure git-lfs is installed first

Git Download 
git lfs install
git clone https://huggingface.co/facebook/vjepa2-vitg-fpc64-256

To skip LFS large-file downloads, use:

Skip LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/facebook/vjepa2-vitg-fpc64-256

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

Install Transformers 
pip install -U transformers torch

Load the model and run inference

Load the model and run inference 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('facebook/vjepa2-vitg-fpc64-256')
tokenizer = AutoTokenizer.from_pretrained('facebook/vjepa2-vitg-fpc64-256')

Model Download

We recommend downloading the model via the ModelScope CLI or SDK.

Guidance：Before downloading, install ModelScope with:

Guidance 
pip install modelscope

CLI Download

Download the full repository

Download the full repository 
modelscope download --model facebook/vjepa2-vitg-fpc64-256

Download a single file to a local folder (e.g. README.md into ./dir)

Download a single file to a local folder (e.g. README.md into ./dir) 
modelscope download --model facebook/vjepa2-vitg-fpc64-256 README.md --local_dir ./dir

See the docs for more CLI options

SDK Download

SDK Download 
# 模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('facebook/vjepa2-vitg-fpc64-256')

Git Download

Make sure git-lfs is installed first

Git Download 
git lfs install
git clone https://www.modelscope.cn/facebook/vjepa2-vitg-fpc64-256.git

To skip LFS large-file downloads, use:

Skip LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://www.modelscope.cn/facebook/vjepa2-vitg-fpc64-256.git

ModelScope 模型页直接下载模型文件；无需将模型文件放在本站服务器。

Notebook Quickstart

Install the ModelScope library

Install the ModelScope library 
pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html

Load the model and run inference

Load the model and run inference 
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

p = pipeline('text-generation', 'facebook/vjepa2-vitg-fpc64-256')

Full Documentation

来源: HuggingFace

---
license: apache-2.0
pipeline_tag: video-classification
tags:

  • video

library_name: transformers
---

V-JEPA 2

A frontier video understanding model developed by FAIR, Meta, which extends the pretraining objectives of VJEPA, resulting in state-of-the-art video understanding capabilities, leveraging data and model sizes at scale.
The code is released in this repository.

<img src="https://github.com/user-attachments/assets/914942d8-6a1e-409d-86ff-ff856b7346ab">&nbsp;

Installation

To run V-JEPA 2 model, ensure you have installed the latest transformers:

bash
pip install -U git+https://github.com/huggingface/transformers

Intended Uses

V-JEPA 2 is intended to represent any video (and image) to perform video classification, retrieval, or as a video encoder for VLMs.

python
from transformers import AutoVideoProcessor, AutoModel

hf_repo = "facebook/vjepa2-vitg-fpc64-256"


model = AutoModel.from_pretrained(hf_repo)
processor = AutoVideoProcessor.from_pretrained(hf_repo)

To load a video, sample the number of frames according to the model. For this model, we use 64.

python
import torch
from torchcodec.decoders import VideoDecoder
import numpy as np

video_url = "https://huggingface.co/datasets/nateraw/kinetics-mini/resolve/main/val/archery/-Qz25rXdMjE_000014_000024.mp4"
vr = VideoDecoder(video_url)
frame_idx = np.arange(0, 64) # choosing some frames. here, you can define more complex sampling strategy
video = vr.get_frames_at(indices=frame_idx).data  # T x C x H x W
video = processor(video, return_tensors="pt").to(model.device)
with torch.no_grad():
    video_embeddings = model.get_vision_features(**video)


print(video_embeddings.shape)

To load an image, simply copy the image to the desired number of frames.

python
from transformers.image_utils import load_image

image = load_image("https://huggingface.co/datasets/merve/coco/resolve/main/val2017/000000000285.jpg")
pixel_values = processor(image, return_tensors="pt").to(model.device)["pixel_values_videos"]
pixel_values = pixel_values.repeat(1, 16, 1, 1, 1) # repeating image 16 times


with torch.no_grad():
    image_embeddings = model.get_vision_features(pixel_values)


print(image_embeddings.shape)

For more code examples, please refer to the V-JEPA 2 documentation.

Citation

```
@techreport{assran2025vjepa2,
title={V-JEPA~2: Self-Supervised Video Models Enable Understanding, Prediction and Planning},
author={Assran, Mahmoud and Bardes, Adrien and Fan, David and Garrido, Quentin and Howes, Russell and
Komeili, Mojtaba and Muckley, Matthew and Rizvi, Ammar and Roberts, Claire and Sinha, Koustuv and Zholus, Artem and
Arnaud, Sergio and Gejji, Abha and Martin, Ada and Robert Hogan, Francois and Dugas, Daniel and
Bojanowski, Piotr and Khalidov, Vasil and Labatut, Patrick and Massa, Francisco and Szafraniec, Marc and
Krishnakumar, Kapil and Li, Yong and Ma, Xiaodong and Chandar, Sarath and Meier, Franziska and LeCun, Yann and
Rabbat, Michael and Ballas, Nicolas},
institution={FAIR at Meta},
year={2025}
}

View official source
Join our Telegram