EvidenceGuard: precise context-based answer selection
2 min readupdated
This model is built on Qwen3.8-27B (an open-architecture large language model) using the peft library (a tool for efficient model fine-tuning). It answers multiple-choice questions strictly based on the provided text, and reports when information is missing. Instead of generating long text, the model analyzes logits (numerical probability scores) at the first answer position, enabling fast and accurate operation.
Key features
- Strictly by context: minimizes hallucinations and relies solely on the provided data.
- No generation: delivers results in a single forward pass without lengthy reasoning.
- High accuracy: outperforms many commercial APIs and reasoning models on the specialized EvidenceGuard test.
- Efficient fine-tuning: uses LoRA (a small add-on over the base model) on top of a powerful foundation.
What you can do
- Document analysis: verify contracts, manuals, and knowledge bases for factual consistency.
- Pipeline integration: embed into automated testing and answer verification systems.
- Test processing: automate multiple-choice questionnaire completion (from 2 to 8 options).
Example
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "Qwen/Qwen3.8-27B"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "EvidenceGuard-Experiments/evidenceguard").eval()Read more
DetailsBenchmarks: 86.5% correct answers on the EvidenceGuard test split (77.6% of groups entirely correct).
Hardware requirements: running requires a powerful GPU (e.g., a single H200 GPU, median latency around 0.12 s per question).
Limitations: trained and tested exclusively on English questions with 2 to 8 answer options; this is not a general-purpose chatbot.










Comments