peshtur
A fast visual choice model with integrated OCR
2 min readupdated
The peshtur model is built on Qwen3.5-4B (a base model with 4.54 billion parameters) and handles multiple-choice visual question-answering tasks. Instead of generating text token by token, the model outputs a calibrated probability distribution across all options (from A to Z) in a single forward pass. The system features a built-in lightweight OCR module (PP-OCRv5) to recognize text in images right during the request. This development will be useful for developers and researchers who need high processing speed for visual data along with accuracy in markup tasks and the analysis of interfaces or documents.
Key features
- Direct output: Evaluates probabilities of answer options in a single forward pass without token generation.
- Integrated OCR: Automatically recognizes text in images using PP-OCRv5 on every request.
- Mixed architecture: Combines full attention layers (SDPA) with Gated DeltaNet recurrent layers.
- Efficiency: Fine-tuned using LoRA across all linear matrices of the language component.
What you can do
- Document and receipt analysis: Apply it to extract key information from forms, receipts, and infographics.
- Content moderation: Use the model to classify memes and check images for policy violations.
- UI tasks: Process screenshots of web interfaces and answer questions about element positioning.
Example
from peshtur import Engine
engine = Engine()
response, _ = engine(
{},
{"answer": {"type": "choice", "instructions": "Which object is closer to the camera?",
"criteria": {"A": "the bicycle", "B": "the bench"}}},
images=["street.jpg"],
)
print(response["answers"]["answer"])Read more
DetailsBenchmarks: 75.44 on the full public Decision Index Vision 0.3 dataset (12,210 rows)
Hardware requirements: GPU with 16 GB VRAM or more (peak consumption around 12.53 GB)
Limitations: R-Bench-M (33.38) and MMMU-Pro vision (41.21) are constrained by the 4B base capabilities
Training details: 1 epoch on 323,264 examples with distillation from three 27B teacher models










Comments