RockAndEgg-313M-v3
a compact Qwen3-based model for calibrated decisions without text generation
2 min readupdated
This 313M-parameter model doesn't write text and lacks a language head—instead, it makes structured decisions with calibrated probabilities. Under the hood is a frozen Qwen3-0.6B-Base backbone pruned from 28 layers down to 10, paired with a compact 268K-parameter SystemOne decision head built by developer moeshawky. You feed it a single state alongside a set of typed questions, and it outputs probabilities across options along with a confidence score. It fits right into pipelines that need predictable category selection, scale-based scoring, or answer abstention instead of open-ended text.
Key features
- Three decision types:
choice(picking a single option),score(rating along an ordered scale), andnoul(deciding whether to answer at all). - Order invariance: shuffling answer choices or questions in the input leaves the resulting probabilities unchanged (0.0 deviation in tests).
- Two precision formats in one repo: the root directory holds the FP32 weights (around 1.25 GB), while the
fp16/folder contains the FP16 version (around 626 MB) with virtually identical calibration. - Frozen backbone: the 312.9M-parameter base weights are bit-for-bit identical to v2—only the decision head itself was fine-tuned on the SargeDev/jev-distill-corpus-v3 dataset.
What you can do
- Route and triage requests: direct incoming queries down the right processing branches using the model's calibrated confidence.
- Score generated content: rate outputs from other AI models or retrieved documents against specific criteria using the
scoretype. - Filter with abstention: use the
noultype in data extraction pipelines so the system holds back from answering when information is missing.
Example
from transformers import AutoModel
# Загрузка основной версии FP32
model = AutoModel.from_pretrained("moeshawky/RockAndEgg-313M-v3", trust_remote_code=True)
# Или облегчённой версии FP16 (626 МБ)
model_fp16 = AutoModel.from_pretrained("moeshawky/RockAndEgg-313M-v3", subfolder="fp16", trust_remote_code=True)
Read more
DetailsMetrics on the test_set_30k holdout split (29,955 examples): Accuracy is 0.5263, while Expected Calibration Error (ECE) sits at 0.03656 for FP32 and 0.03677 for FP16.
Training the head took 1 epoch (5,002 steps, 80,000 examples) on two NVIDIA Tesla T4 GPUs in roughly 57 minutes.
Limitations: the model only supports English, cannot generate text, and is not a general-purpose classifier outside the logic of its training corpus; using it requires attributing the SargeDev/jev-distill-corpus-v3 dataset (Apache-2.0 license).










Comments