SWAM
scalable world action models for robotics
1 min readupdated
SWAM (Scaling World Action Models with Cross-Modal Alignment) is a family of checkpoints for robotics and action control tasks, utilizing cross-modal alignment. The repository contains pre-trained weights as well as versions fine-tuned on RoboTwin data. The model helps connect visual perception of the world with robot control actions and relies on Wan and UMT5 components. The checkpoints are suitable for researchers and engineers working in robotics and environmental simulation.
Key features
- Cross-modal alignment: efficient linking of visual data and robot actions.
- Two checkpoint variants: pre-trained weights and a version fine-tuned on RoboTwin Clean simulation data are available.
- SafeTensors format: weights are split into shards (sharded Safetensors), with each checkpoint weighing approximately 27.98 GB.
- Open ecosystem: full integration with the project's datasets and source code.
What you can do
- Run simulations: use the checkpoints to evaluate and train robotic agents in the RoboTwin environment.
- Fine-tune models: apply SWAM pre-trained weights for custom robotics tasks and hardware adaptation.
Example
Downloading a pre-trained SWAM checkpoint via the command line:
pip install -U huggingface_hub
hf download FreDERicia/SWAM --include "pretrain/*" --local-dir ./SWAMRead more
DetailsCheckpoint size: ~27.98 GB each
File format: Sharded Safetensors
Architecture: Requires preliminary preparation of Wan and UMT5 components
Benchmark suite: RoboTwin Clean benchmark










Comments