PLWM
point-latent world model for robotics
2 min readupdated
PLWM (point-latent world model) is a research checkpoint for robotics that extends the Wan2.2-TI2V-5B architecture with a 3D point stream and a built-in action decoder. Trained on real-world, four-camera bimanual robot data, it predicts both visual transitions and manipulator actions. Under the hood, it combines the Wan2.2 video diffusion transformer with a frozen point autoencoder, making it a solid base for researchers and engineers working on robotics and world models.
Key features
- Built on Wan2.2-TI2V-5B with an added 3D point-latent stream.
- Integrated action decoder for bimanual end-effector actions running at 10 Hz.
- Trained on real-world bimanual robot episodes recorded from four camera angles.
- Includes the final training checkpoint from 8 H100 GPUs (step 10,000) along with the package code.
What you can do
- Run a CPU model-loading smoke test using the included script.
- Set up inference by linking public Wan2.2-TI2V-5B weights and the umt5-xxl text encoder.
- Use the package codebase and Hydra configs to run your own world-model experiments.
Example
pip install torch hydra-core omegaconf einops imageio rich safetensors transformers
cd world_model && python load_test.py # добавьте --ema для весов EMARead more
DetailsBuilt on Wan2.2-TI2V-5B and a frozen SE(3)-latent point AE (144 slots of 128 channels each).
Dataset: 46 training and 5 validation episodes of a bimanual robot captured at 10 Hz.
Optimization: DDP across 8x H100, global batch size 16, lr 1e-4, 8-bit AdamW, bf16.






Comments