NeuroRVQ-LLM-Qwen2.5-7B
EEG-to-text decoding with an LLM
2 min read
This model tackles brain-computer interface (BCI) decoding, translating electroencephalography (EEG) signals recorded during reading back into coherent sentences. Built on the Qwen/Qwen2.5-7B-Instruct large language model (LLM), it is fine-tuned with LoRA (a lightweight adapter for efficient fine-tuning) and a specialized Residual Vector Quantization (RVQ) method tailored for brain data. Trained on the ZuCo datasets and evaluated on unseen subjects without teacher forcing (ground-truth prompting during generation), it is geared toward researchers working on neural interfaces and cognitive science.
Key features
- Qwen2.5-7B backbone: Uses the
Qwen/Qwen2.5-7B-InstructLLM as its text generator. - Connectome-Guided RVQ: Applies vector quantization for stable cross-subject EEG signal decoding.
- Subject-Independent: Evaluated zero-shot on unseen subjects, consistently outperforming random noise baselines.
- Complete weights included: Ships with stage checkpoints, the
BrainToLLMProjector, and a PEFT LoRA adapter.
What you can do
- Integrate into BCI pipelines: Use the pre-trained weights and configs from the repo to reproduce EEG decoding experiments.
- Evaluate generation quality: Run text decoding on your own neuroimaging datasets or execute the repository's built-in benchmarks.
- Explore brain-to-LLM projections: Inspect how the 2-layer projector maps discrete brain tokens into the language model's embedding space.
Example
Comparison between a real sentence (Ground-Truth) and the decoded output (NeuroRVQ-LLM Decoded Output) on an unseen subject:
Ground-Truth: Presents a good case while failing to provide a reason for us to care beyond the very basic dictums of human decency.
Decoded Output: Despite the holes in the story and the somewhat predictable plot, moments of the movie caused me to jump in my chair...
Read more
Details• Datasets: ZuCo 1.0 & ZuCo 2.0 (18,276 trials across 30 unique subjects).
• Metrics (Stage 1 Zero-Shot Retrieval): R@1 9.90%, R@5 35.08%, R@10 56.09%, MRR 23.14% (on 1,313 sentences).
• Metrics (Stage 2 Open-Vocabulary): BLEU-1 19.60%, BLEU-4 9.89%, ROUGE-1 F1 19.58%, WER 90.85%.
• Architecture: Rank-16 BF16 LoRA + 50% Scheduled Target-Word Dropout on Qwen2.5-7B-Instruct.










Comments