Llama-V-JEPA-LM-Qwen3-30B-A3B
a powerful video-language model for scene dynamics analysis
2 min readupdated
The model combines a frozen V-JEPA 2.1 (ViT-G) video encoder with a Qwen3-30B-A3B MoE language model. It is designed to understand motion, direction, speed, and object states in video clips. Thanks to a dedicated spectral bridge, each clip is converted into 1,024 visual tokens regardless of the original resolution. The new model outperforms the base Qwen3-VL-30B-A3B and other open alternatives across a range of video benchmarks while retaining the strong mathematical and text skills of the original language backbone.
Key features
- Outperforms open models of a similar class in video comprehension tests (69.49 average across 11 benchmarks).
- Trained using counterfactual supervision for precise tracking of scene dynamics.
- Efficient resolution handling: the number of visual tokens is fixed at 1,024 and does not depend on the original video resolution.
- Retains the strong mathematical and text capabilities of the base language model (GSM8K 88.7).
What you can do
- Analyze video fragments of any length using the built-in chat assistant to answer questions about the content.
- Set up multiple-choice questions for accurate evaluation of logic and events in the video.
- Run analysis for a specific video timecode using the start_sec and end_sec parameters.
- Perform autonomous video encoding using the encode_video function and ask different questions of the same cache.
Example
import torch
from transformers import AutoModel
model = AutoModel.from_pretrained(
"kkt94/Llama-V-JEPA-LM-Qwen3-30B-A3B",
trust_remote_code=True, dtype=torch.bfloat16,
).to("cuda").eval()
print(model.chat("my_video.mp4", "Describe what happens in the video."))
Read more
DetailsArchitecture: MoE (30.53B parameters, 8 active experts out of 128) + V-JEPA 2.1 encoder (1.85B).
Hardware requirements: about 61 GB VRAM to process a single test clip.
License: Apache-2.0 (encoder: MIT, training data: Llama 3.2 Community License).










Comments