track_v3_dual-image
a driving policy for F1TENTH mini-racers
2 min readupdated
This trained control policy is built for autonomous mini-robots on the f1tenth platform. It takes simultaneous feeds from two onboard cameras (camera1 and camera2) along with the vehicle's current state, outputting steering and throttle commands to run laps clockwise or counterclockwise. Under the hood is the compact 0.45B-parameter VLA model (Vision-Language-Action — an AI model connecting vision, text commands, and physical actions) lerobot/smolvla_base by Szymon-Ilnicki. Unlike the base model, this version is fine-tuned specifically for dual-stream autonomous driving using the LeRobot framework. It is useful for robotics developers and enthusiasts working with the F1TENTH platform or building custom visual navigation policies on SmolVLA.
Key features
- Dual-camera vision: processes two 424×240 video streams (
camera1andcamera2) alongside a 2-dimensional robot state vector. - Bidirectional driving: trained to follow text prompts for running track laps both clockwise and counterclockwise.
- Compact footprint: the 0.45B-parameter SmolVLA backbone is built to run on consumer hardware without heavy server accelerators.
- LeRobot integration: fully compatible with the standard
lerobot-rolloutpipeline for quick deployment on a physical robot.
What you can do
- Run autonomous laps on a physical
f1tenthchassis by hooking up two cameras and specifying the driving direction in a text prompt. - Use the checkpoint as a ready-made starting point for fine-tuning on your own track or camera setup via
lerobot-train. - Explore how multimodal SmolVLA policies behave in practice on visual vehicle-control tasks.
Example
lerobot-rollout \
--strategy.type=base \
--robot.type=f1tenth \
--robot.port= \
--robot.cameras="{ camera1: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, camera2: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30}}" \
--policy.path=Szymon-Ilnicki/track_v3_dual-image \
--task="complete laps around the track anticlockwise" \
--duration=60
Read more
DetailsTraining: 20,000 steps (batch size 8, lr 0.0001, adamw optimizer) on the
local/track_v3 dataset of 12 episodes (4,870 frames at 10 FPS).Framework version: LeRobot 0.7.0, Apache 2.0 license.
Limitations: real-robot test results are not yet listed in the model card; camera names at runtime must strictly match the keys used during training.










Comments