AutoDrive-OPD-E057O-student
Qwen3-VL Based Autopilot Model
2 min readupdated
This model is a specialized fine-tune of the Qwen3-VL-4B-Instruct vision-language model, trained for autonomous driving tasks in the NAVSIM simulator. It takes an image from the front camera and a text prompt with driving parameters (navigation command, current speed, acceleration, angular velocity, and past trajectory) as input, and outputs a block containing the predicted trajectory for the next 4 seconds. The training was performed using on-policy distillation based on two expert reinforcement learning (RL) models. This development will be useful for researchers in autonomous transport and computer vision.
Key features
- Qwen3-VL Architecture: Full-parameter training of a 4-billion-parameter model with support for image and text processing.
- Policy Distillation: Leveraging data from two RL-driven experts to improve stability.
- Precise Trajectory Prediction: Generation of 8 coordinate points [x, y, yaw] at 0.5-second intervals in the vehicle's current coordinate system.
- NAVSIM Evaluation: Tested on 12,146 benchmark test scenes with zero parsing errors.
What you can do
- Testing in NAVSIM: Evaluate autonomous driving performance on simulation datasets derived from nuPlan.
- Autonomous Systems Research: Use as a baseline for experiments with end-to-end trajectory planning.
- Pipeline Integration: Apply generation and evaluation scripts to analyze multimodal agent performance on road scenes.
Example
The model takes a frame from the CAM_F0 front camera and a string prompt containing current speed, acceleration, and past agent poses as input, and then generates a tag with the coordinates of the motion trajectory points.
Read more
DetailsBenchmarks: navtest PDMS — 0.9227, navtest EPDMS (guarded) — 0.9125, navhard EPDMS — 0.3022.
Inference parameters: greedy decoding, max_new_tokens=512, min_pixels=262144, max_pixels=1048576.
Base model: Qwen3-VL-4B-Instruct (Apache-2.0 license), NAVSIM / OpenScene (nuPlan) dataset.










Comments