ericflo/stet-glide-pairwise-adapter
a text educational value evaluation paradigm
2 min readupdated
A new rank-8 LoRA (small add-on layer for a base model) and decision head built on top of the fastino/GLiDE-0.4B model. This combination evaluates the educational value and style of web passages, helping select quality replacements for prose, pairwise rank curricula, and selectively repeat data in the Stet project. Training took place in two stages: first, fine-tuning on document quality and style, followed by adding a pairwise objective function. The model will be useful for researchers and developers filtering and evaluating the quality of large text corpora and educational materials.
Key features
- Rank-8 LoRA and an additional decision head for text evaluation.
- Trained on data from FineWeb-Edu, FineWeb-C, OpenWebMath, and Python documentation.
- Improved agreement on held-out historical pairs from 76.95% to 79.05%.
- Integration with the base encoder via the transformers.AutoModel library.
What you can do
- Evaluate the educational value and style of web passages in text pipelines.
- Pairwise rank educational texts and select quality replacements.
- Set up selective data repetition to improve training corpora.
Example
Loading the base encoder using transformers.AutoModel and attaching the adapter from the adapter/ folder
Read more
Details• Metrics: accuracy on held-out pairs increased from 76.95% to 79.05%.
• Architecture: rank-8 LoRA, base model fastino/GLiDE-0.4B.
• Components: adapter in adapter/, quality head in lora-head.safetensors.
• Training data: FineWeb-Edu, FineWeb-C, OpenWebMath, Python docs.










Comments