task1task2_subtask_rel
multi-task robotic arm control model
2 min readupdated
Built on the pi0.5 architecture (a versioned robot control model from Physical Intelligence / LeRobot), this multi-task model is trained for three manipulation tasks: inserting a board into a module, grasping a cable, and moving the module to the center of the table. It uses a vision-language backbone (VLM) to predict intermediate subtasks before generating control actions. Combining several datasets, the model processes feeds from three cameras alongside sensor state data (a 24-dimensional vector), making it a solid fit for robotics researchers and developers working with the LeRobot framework.
Key features
- Supports three different robotic arm scenarios through a single pipeline.
- Uses subtask prediction for accurate action planning.
- Trained on combined demonstrations (GRC, Yonsei, Hyundai) with a batch size of 128.
- Leverages bf16 precision and gradient checkpointing for stable training.
What you can do
- Run model inference to control a robotic arm for assembly and mounting tasks.
- Test multi-task prompting to switch between inserting a board and grasping a cable.
- Integrate the model with the LeRobot plugin to customize it for your own robotics testbeds.
Example
# pip install git+[https://github.com/HDR-SRL/lerobot_policy_pi05_subtask@multitask](https://github.com/HDR-SRL/lerobot_policy_pi05_subtask@multitask)
import lerobot_policy_pi05_subtask
from lerobot.policies.factory import make_pre_post_processors
from lerobot_policy_pi05_subtask import PI05SubtaskPolicy
policy = PI05SubtaskPolicy.from_pretrained("chomeed/task1task2_subtask_rel_chunk30_sirius_bs128_multitask_fromR2mixed_1118_20k")
pre, post = make_pre_post_processors(policy.config, pretrained_path="chomeed/task1task2_subtask_rel_chunk30_sirius_bs128_multitask_fromR2mixed_1118_20k")
batch = pre({"observation.state": state, "observation.images.d405_left": ..., "task": "insert the cable into the control module"})
actions = post(policy.predict_action_chunk(batch))
Read more
DetailsTrain loss: 0.002 at step 20,000
Architecture: LeRobot 0.6.2 pi0.5_subtask + lerobot_sirius
Inputs: 3 cameras and 24-dimensional state
Output: 30-step action sequences (11-D)










Comments