CGVM-R2.6-Qwen3VL-8B-step600
checkpoint for accurate analysis of short and long videos
2 min readupdated
An intermediate LoRA adapter (a lightweight add-on on top of the base model) checkpoint for the multimodal Qwen3-VL-8B-Instruct model, created by XiangnanW. It handles video understanding and multiple-choice Video QA tasks, fixing weaknesses from the previous R2.5 release when dealing with short clips, motion direction, speed, and repetition counting. Built for researchers and video analytics developers who need consistent accuracy across both short clips under 24 seconds and long video streams.
Key features
- Dense sampling for short videos: For clips shorter than 24 seconds, the model uniformly extracts a full 16 frames across the entire clip instead of sparse 1 FPS (frames per second) sampling, while keeping the existing handling logic for streams longer than 24 seconds.
- Support for 2–5 multiple-choice questions: The inference pipeline uses log_softmax calibration that relies on the LoRA prediction when confidence is high and falls back to the base model only in ambiguous cases.
- Decoding failure protection: Adds automatic fallback to sequential reading via cv2 for videos with invalid negative timestamps (PTS), preventing hangs during processing.
- Expanded 28,000-sample dataset: Adds 8,000 question-answer pairs from PerceptionTest, MotionBenchCaption, and SSv2 to the base data, focusing on action order, camera angle, speed, and reverse playback.
What you can do
- Run video understanding benchmarks: Test the step-600 weights alongside the updated repository code on TempCompass and MotionBench.
- Integrate into a Video QA pipeline: Use the updated frame buffering mechanism from
src/to analyze short video clips and streaming video. - Compare against R2.5: Measure accuracy gains on motion direction, speed estimation, and counting repeated actions.
Example
torchrun --nproc_per_node=8 scripts/evaluate_adapted.py \
--variant R2 --benchmark tempcompass \
--reader-dir checkpoints/R2/reader \
--output-root results/R2_6_step600 --resumeRead more
DetailsWeight status: intermediate snapshot step-000600 out of 875 steps (trained on 19,200 of 28,000 examples, 68.57%), 677 MB adapter size.
Training parameters: 4×H100, global_batch=32, lr=6e-5, LoRA r=64, alpha=64, kl_weight=0.10.
Preliminary benchmarks (R2.5 + new R2.6 pipeline): TempCompass (n=160) rose from 73.75% to 75.62%, MotionBench (n=156) from 57.05% to 61.54%.
Limitation: All fixes require the updated inference code from the
src/ folder or the cgvm-r26-src-v1.tar.gz archive; legacy R2.5 code will not activate the new mechanisms.









Comments