PEARL-Qwen3-VL-8B-LoRA
an adapter for streaming video analysis using custom concepts
2 min readupdated
This is a LoRA (a lightweight add-on for a base model) for the Qwen3-VL-8B-Instruct vision-language model, trained as part of the PEARL framework. It is built for personalized streaming video understanding, letting users ask questions about custom concepts defined at runtime. The model handles both questions about the current video segment (Real-Time) and memory-based queries from earlier segments (Past-Time). Applied during the final response stage alongside a retriever, the adapter boosts overall accuracy.
Key features
- Trained on the PSVU-10K dataset, which contains 10,000 questions across videos from 155 sources.
- Improves response accuracy on PSVU-Bench across all categories, including behavior, objects, and locations.
- Contains 15.3M trainable parameters with rank 16 and alpha 32.
- Released under the permissive Apache-2.0 license.
What you can do
- Use it in the PEARL pipeline to answer questions about custom objects in streaming video.
- Load the adapter weights in vLLM with LoRA support enabled to generate final responses.
- Integrate the adapter via
transformersandpeftfor custom video processing pipelines.
Example
from vllm import LLM, SamplingParams
from huggingface_hub import snapshot_download
from vllm.lora.request import LoRARequest
llm = LLM(model="Qwen/Qwen3-VL-8B-Instruct", enable_lora=True, max_lora_rank=16)
answer_lora = LoRARequest("pearl-answer", 1, snapshot_download("zyh200727/PEARL-Qwen3-VL-8B-LoRA"))
sampling = SamplingParams(temperature=0.0)
helper_outputs = llm.generate(helper_inputs, sampling)
answer_outputs = llm.generate(answer_inputs, sampling, lora_request=answer_lora)
Read more
DetailsBase model: Qwen/Qwen3-VL-8B-Instruct
Overall accuracy on PSVU-Bench: 61.41% (with adapter)
Training: 9,140 steps, AdamW, batch size 8, BF16
Weights: FP32, 61.4 MB










Comments