qwen3.5-2b-receipt-invoice-extract-lora
receipt and invoice data extraction to JSON
2 min readupdated
This is a LoRA (a lightweight add-on for a base model) for the Qwen/Qwen3.5-2B vision-language model, created by zackyalgiffari. It takes a photo or scan of a receipt or invoice and converts all the information from the image into a single structured JSON object following the Donut dataset schema. Unlike the base model, which invents its own field names out of the box and scores 0.000 on Field F1, this adapter strictly follows the target structure and outputs 100% valid JSON. It is a great fit for developers automating bookkeeping, expense tracking, and primary document digitization.
Key features
- Strict output structure: returns data matching the CORD (
menu[].nm,total.total_price) and invoices-donut-data-v1 (header,items[],summary) dataset schemas. - High value accuracy: across a test set of 126 documents, the adapter boosted
Value F1from 0.462 to 0.916, andField F1from 0.000 to 0.886. - Supports two document types: switching between receipt and invoice modes takes just a single word in the system prompt (
receiptorinvoice). - Open license: the adapter is released under Apache-2.0, matching the base Qwen3.5-2B model.
What you can do
- Automate receipt tracking: plug the model into a pipeline processing supermarket and cafe photos to quickly parse line items and totals.
- Digitize invoices: use it for initial invoice parsing with automatic extraction of headers, item lists, and summaries.
- Deploy a local service: merge the weights into the base model via
merge_and_unload()from thepeftlibrary for lightweight single-GPU inference.
Example
from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor
base = "Qwen/Qwen3.5-2B"
model = AutoModelForImageTextToText.from_pretrained(base, dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, "zackyalgiffari/qwen3.5-2b-receipt-invoice-extract-lora").merge_and_unload()
processor = AutoProcessor.from_pretrained(base)
# В промпте передаётся: "Extract all fields from this receipt image. Respond with only a JSON object."
Read more
DetailsMetrics (100 CORD receipts and 26 invoices): 100% valid JSON, Field F1 — 0.886 (Lenient F1 — 0.921), full document exact match — 38.9% (48.0% for receipts and 3.8% for invoices).
Hardware and training: trained for 2.5 hours on a single NVIDIA L4 (24 GB) with a peak usage of 11.7 GiB VRAM on 800 CORD v2 receipts (Indonesian) and 425 invoices.
Limitations: supports only two rigid key schemas; occasional typos or dropped characters may occur in long codes (such as IBANs), names, and totals, so numbers require verification.










Comments