trossen-bin-picking-act-cup-real
robot control policy for cup picking
2 min readupdated
A compact 50M-parameter model from the Kite team built on the ACT architecture (Action Chunking with Transformers, an algorithm that predicts sequences of robot actions at once) for the bimanual WidowX AI robotic station. It takes feeds from four cameras alongside the current positions of 14 joints, outputting target coordinates for all 14 joints 100 steps ahead. Trained on real-world robot motion recordings from Trossen Robotics' open bin_picking dataset, this policy outperforms counterparts trained solely in simulation. It is built for robotics developers and researchers working with the LeRobot framework.
Key features
- Multi-sensor input: simultaneously processes streams from four cameras and the current positions of the station's 14 actuators.
- Action chunking: generates a motion trajectory 100 steps ahead across 14 joints in a single forward pass.
- Trained on real-world data: weights were trained on 18 real recordings of cup picking (7,318 frames) rather than synthetic data.
- Task-specific design: takes no text prompts and is strictly tuned to grasp a red cup and place it into a clear bin.
What you can do
- Run in simulation: test bimanual WidowX AI control under varying lighting conditions and with distractor objects on the table.
- Benchmark training approaches: use the checkpoint as a baseline when comparing policies trained on real trajectories versus simulated
trossen-bin-picking-simdatasets. - Fine-tune for custom tasks: use these weights as a starting point in LeRobot 0.6 and newer for object manipulation experiments.
Example
from lerobot.policies.act.modeling_act import ACTPolicy
policy = ACTPolicy.from_pretrained("kiteml/trossen-bin-picking-act-cup-real")
Read more
DetailsSimulation results: 33 successful picks out of 80 (10/20 under normal lighting, 8/20 in dim light, 8/20 in warm light, and 7/20 with extra objects on the table), beating the best simulation-trained model (26/80).
Training: 30,000 steps with a batch size of 8 on a single A100 GPU took roughly 1 hour and 50 minutes.
Limitations: evaluated only in simulation (not yet tested on physical hardware) and only succeeds when grasping with the left arm.
License: Apache-2.0.










Comments