VivekSai07/xvla_hfteam_box_middle
fine-tune for robotic arms
2 min read
The X-VLA model (0.88B) has been fine-tuned on teleoperation demonstrations for synchronized dual-arm operation using SO-101 robotic arms. The architecture processes data from three cameras, 12 joint positions, and a text prompt, outputting control actions. This fine-tune (adapting model weights for a specific task) was created during a hackathon for cooperatively lifting a box and moving it to the center. The solution is useful for engineers and researchers working with the LeRobot library.
Key features
- Base model: lerobot/xvla-base (0.88 billion parameters).
- Input data: three cameras, 12 joint positions, and task text.
- Control: generation of 1-second action chunks for two robotic arms.
- Training: 7,000 steps with a batch size of 16 and bf16 autocast (automatic mixed precision data type).
What you can do
- Evaluate the model's behavior on dual SO-101 robotic arms.
- Test cooperative box transfer via a text instruction.
- Apply the weights for multi-arm system experiments.
Example
lerobot-rollout --strategy.type=base \
--policy.path=VivekSai07/xvla_hfteam_box_middle \
--robot.type=bi_so_follower \
--task="Lift the box with both arms and place it in the middle" --duration=30Read more
DetailsTraining: 41 teleoperation episodes (20,556 frames at 30 fps).
Metrics: training loss dropped from 0.79 to 0.003 at step 7,000.
Quality: average error on training frames is 0.9 degrees.
Limitations: success rate on real hardware has not been measured yet.










Comments