Agnes-Video-H3-MoE
sparse MoE add-on for video generation based on MiniMax-H3
2 min readupdated
This is an experimental checkpoint that turns dense network layers in blocks 15–34 of the MiniMax-H3 FL2VA model into a sparse MoE (Mixture of Experts) architecture at the token level. For each token, the model runs one shared expert and selects one of eight private experts, with different tokens in the same video able to activate different paths. The result is video and audio generation that supports conditional generation using the first, last, or both frames. The repository ships as a composite package with base model weights, a routing map, and trained LoRA weights (a small add-on to the base model).
Key features
- Sparse MoE architecture: conversion of 20 out of 50 diffusion blocks into a token-level Top-1 structure with shared and private experts.
- Multimodal output: simultaneous generation of high-quality video footage with audio accompaniment.
- Flexible control: support for working with the first and last frames as conditions.
- Reference loader: includes ready-to-use DiffSynth scripts for inference and package verification.
What you can do
- Generate a high-quality 10-second video clip using the reference Python inference script.
- Run text-to-video generation by adding starting or ending frames as reference images.
- Check the integrity of the weight package and routing configuration using an offline validation script.
Example
hf download Agnes-AI/Agnes-Video-H3-MoE --local-dir E8T1-shared1
cd E8T1-shared1
python deploy/check.py
python deploy/infer.py \
--package . \
--prompt "A cat walks across a piano." \
--out sample.mp4 \
--height 480 --width 832 --frames 124 --steps 50 --seed 42Read more
DetailsBased on the MiniMax-H3 FL2VA model in BF16 format.
The total repository size is around 135 GB.
Requires a powerful CUDA-enabled graphics card, PyTorch, and significant system memory with CPU offloading.
Training took 60 steps on 128 examples.














Comments