SmolVLA Multi-Object Pick-and-Place
robotic arm control model
2 min readupdated
This checkpoint is a fine-tune of the lerobot/smolvla_base weights on a 200-episode dataset for pick-and-place tasks (such as "pick up the brown cube and place it in the black box"). The model predicts coordinates for six joints on a robotic manipulator arm autonomously, removing the need for a leader arm. It is built for researchers and developers working with the LeRobot framework and hardware like the SO-101.
Key features
- Native LeRobot model: uses the 0.45B-parameter SmolVLA architecture for autonomous vision-based robotic arm control.
- Dual-camera setup: combines an overhead view with a wrist-mounted gripper camera for accurate positioning.
- Trained on NVIDIA H100: trained for 20,000 steps using BF16 mixed precision.
- Normalized positions: outputs relative joint coordinates roughly in the [-100, 100] range instead of degrees.
What you can do
- Run the arm autonomously: connect a compatible manipulator like the SO-101 and execute autonomous pick-and-place tasks.
- Configure camera feeds: connect two 640x480 cameras and map their streams to the model inputs.
- Test inference: run an autonomous rollout script test without recording new datasets.
Example
Sample command to run an autonomous test rollout on the robotic arm:
lerobot-rollout \
--strategy.type=base \
--inference.type=sync \
--policy.path=Daankrol/smolvla-multiobj-lerobot-20k \
--policy.device=cuda \
--policy.load_vlm_weights=false \
--robot.type=so101_follower \
--robot.port="$FOLLOWER_PORT" \
--robot.use_degrees=false \
--task="pick up the brown cube and put it in the black box"Read more
DetailsTraining steps: 20,000
Batch size: 64
Final training loss: 0.034
Final evaluation loss: 0.4978
Requirements: Python 3.12, LeRobot 0.6.0, CUDA-compatible NVIDIA GPU










Comments