docstructure-4b-v64-m3k-cut24-fullm
document structure analysis model
2 min readupdated
Built on Qwen3.5-4B (an open-source language model from the Qwen team), this model analyzes document structure by taking page images and text regions as input and outputting a hierarchical heading tree. It was created by pruning the base model down to 24 blocks, running an intermediate LoRA (a lightweight adapter for fast fine-tuning) pass, merging the weights, and performing full fine-tuning. It is well suited for developers and researchers working on complex document parsing and automated table-of-contents extraction.
Key features
- Specialization: builds a parent tree for every layout region in a document.
- Architecture: a pruned version of Qwen3.5-4B retaining the first 24 language blocks.
- Training: combines initial rank-16 LoRA training with a subsequent full weight update of the language backbone.
- Output format: uses special single-token vocabulary markers to generate structural links quickly.
What you can do
- Analyze documents: process multi-page PDFs or scans to automatically generate tables of contents.
- Integrate into pipelines: embed the model into OCR (optical character recognition) and structure-extraction workflows for scientific or legal documents.
Example
The model relies on standard configuration files alongside template registration scripts like register_lz.py and common.py to handle page images and tokens.
Read more
DetailsValidation loss: 0.031 (down from 0.036 after the merge).
HeadTEDS benchmarks (MarathonDocBench / MPDocBench): 57.06 / 61.21 (draft-head, bf16).
Training data: 185 synthetic documents with context windows ranging from 1 to 90 pages.










Comments