plwm FurnitureBench world model
a robotics world model built on Wan2.2
2 min readupdated
This world model uses a point-based latent representation for robot manipulation, built on top of Wan2.2-TI2V-5B. It predicts future point clouds of the robot and scene alongside video. The model was trained on tracked point clouds from FurnitureBench (including square table and other tasks) with a resolution of 4096 points per frame and 32 history steps. This development is useful for researchers and engineers working in robotics, computer vision, and physical interaction simulation.
Key features
- Built on Wan2.2-TI2V-5B (DiT + VAE) and initialized from scratch.
- Trained for 100,000 steps with a global batch size of 64 and the AdamW optimizer.
- Uses an SE(3) latent point autoencoder to encode point clouds into a latent space.
- Includes exponential moving average (EMA) weights for evaluation and a point-to-action decoder.
What you can do
- Explore scene dynamics and point cloud forecasting in robotic manipulation tasks.
- Use the
ema.ptcheckpoint together with theencoder/ae_wmstats.ptencoder for model performance evaluation. - Apply the
pts_actdec.ptdecoder to map point clouds to robot actions.
Example
To work with the model, use the config.yaml configuration file, the ema.pt weights, and the encoder/ae_wmstats.pt encoder as specified in the repository documentation.
Read more
DetailsArchitecture: Wan2.2-TI2V-5B (DiT + VAE)
Training: 100,000 steps, global batch 64, AdamW lr 1e-4, cosine schedule, bf16, EMA 0.999
Included files: config.yaml, ema.pt (~10 GB), encoder/ae_raw.pt, encoder/ae_wmstats.pt, pts_actdec.pt (~50 MB)










Comments