iclr27_release
a suite of 15 robotics policies powered by V-JEPA 2.1
2 min readupdated
An anonymous weight release (likely tied to an ICLR 2027 submission) featuring 15 robot control models (Stage-2 policies) and 3 paired action codecs. It uses a frozen V-JEPA 2.1 ViT-L/16 vision backbone (at 384 resolution), while the policy itself takes feeds from three cameras (first_person, wrist_left, wrist_right) alongside the robot's proprioceptive state to predict a 30-step action sequence. The release includes a primary multitask model (main_multitask), 11 specialized single-task policies for single-arm and bimanual manipulation, and 3 ablation models for comparing motion-encoding methods.
Key features
- Multitask and single-skill checkpoints: The main model was trained for 120K steps across all tasks simultaneously, alongside 11 individual checkpoints (30K–50K steps) tailored to specific actions.
- Three action codec variants: Beyond the primary encoder/decoder, the release includes paired codecs for
Action-MAEandAction-iBOT(supportingsiglipandibottargets), which compress actions into 512-dimensional latent tokens. - Two inference modes: Run full RGB inference (requires downloading the V-JEPA 2.1 weights separately) or quickly benchmark on precomputed features (
--features-only) without the vision backbone. - Full inference reproducibility: Weights are saved in their original precision (including EMA networks) and come bundled with normalizers, latent-space statistics, and checksum verification scripts.
What you can do
- Run policy inference in a simulator or on real hardware: Feed in frames from three RGB cameras and a D324 state vector to output ready-to-use
[30, 324]action chunks. - Use the action encoders standalone: Compress motion trajectories into compact latent representations via
load_codec_networksfor your own robotics experiments. - Compare action representations: Benchmark the differences between raw actions (
raw_action),action_mae, andaction_iboton precomputed features.
Example
from load_release import load_policy
policy = load_policy(
"main_multitask",
device="cuda",
vjepa_weights="../vjepa2_1_vitl_dist_vitG_384.pt",
)
# policy.predict(observation).action возвращает чанк действий [30, 324]Read more
DetailsReference scores (50 episodes per task, seed-0):
main_multitask — 357/550 successes, ablations/action_mae — 252/550, ablations/action_ibot — 240/550, ablations/raw_action — 117/550.Top single-task policies: watering a plant (
water_plant) — 48/50, mouse clicking and using tongs — 41/50 each, bimanual filming — 39/50; the hardest tasks are bimanual assembly (1/50) and unlocking an iPad (8/50).Requires Python 3.10+ and PyTorch; external V-JEPA 2.1 weights are downloaded separately via
download_vision.py.The archive does not include the simulator, datasets, or optimizer states (inference weights only). License: CC BY-NC 4.0 (non-commercial use only).










Comments