smolvla-keys101-22keys
VLA model for playing keys
2 min read
A compact VLA (Vision-Language-Action — an AI model that takes an image and text prompt as input and outputs motor actions) robotic model trained to control a so_follower robotic arm to press 22 musical notes from C3 to A#4. Built on lerobot/smolvla_base (0.45B parameters) and fine-tuned using the LeRobot framework on 755 episodes with three cameras. Ideal for robotics enthusiasts and owners of SO-101 robotic arms who want to run control policies on regular consumer hardware.
Key features
- Understands text prompts to press 22 specific notes (for example,
press D4,press G#3,press A#4). - Accepts a 6-value robot state vector and streams from three cameras at 256x256 pixel resolution as input.
- The compact 0.45B parameter size allows running the model on consumer hardware.
- Trained on the
PawelLach/so101-keys101-22keysdataset (755 episodes, 39,790 frames at 30 FPS).
What you can do
- Run
so_followerrobot control via thelerobot-rolloutCLI utility to press specified notes based on text prompts. - Use the checkpoint or training configuration in LeRobot to experiment with imitation learning on keyboard instruments.
Example
lerobot-rollout \
--strategy.type=base \
--robot.type=so_follower \
--robot.port= \
--policy.path=PawelLach/smolvla-keys101-22keys \
--task="press D4" \
--duration=60
Read more
DetailsBase model: lerobot/smolvla_base (0.45B parameters), Apache-2.0 license.
Training: 6,000 steps, batch size 64, AdamW optimizer, learning rate 0.0001, LeRobot 0.7.1.
Limitations: evaluation results on a real robot have not yet been published by the author.










Comments