Metis Hybrid Olala mini
a hybrid model for structured prompts
2 min read
This model is a LoRA adapter (a small add-on over a base model) and a calibrator built on top of the frozen base OVHaiLLM/Olala-v1.0-mini-7A1B (6.8B total parameters, roughly 1B active per token) by GreenPT. It is designed to handle TypeSafe Jev format prompts via /v1/systemone with calibrated probabilities while preserving standard chat features. Unlike the standard base, it features an added decision head and frame-by-frame calibrator.pt calibrator, and uses a shortened system prompt for requests. The model is useful for developers who need structured solutions with numerical confidence scoring.
Key features
- Hybrid architecture: combines a LoRA adapter (rank 16), three trained rows of special tokens, a decision head, and a
calibrator.ptcalibrator. - Probability calibration: outputs calibrated confidence scores for answers to prompts, reducing the discrepancy between confidence and actual accuracy.
- Three operation modes: supports answer-only output (
systemone), a combined mode with explanations (both), and a standard language model mode (text). - Ready-to-use wrapper: includes the
integrated_decisions_v2loading code and thescripts/serve.pyserver startup script.
What you can do
- Deploy an API server: spin up a local service for processing POST requests to
/v1/systemonevia theserve.pyscript inside the OVHai LLM Docker image. - Integrate into Python: connect to the model using the
DecisionLMclass for programmatic processing of states and questions. - Get decision explanations: run in
mode="both"to receive textual justifications for each answer alongside the probabilities. - Use in chat mode: conduct a regular dialogue using the standard system prompt of the Olala mini base model.
Example
import os, sys
from huggingface_hub import snapshot_download
os.environ["IDV2_BACKBONE_REVISION"] = "0fde460f37d0fe2c390619bb5729c9d81dc89091"
path = snapshot_download("GreenPT/metis-hybrid-olala-v1.0-mini")
sys.path.insert(0, path)
from integrated_decisions_v2 import DecisionLM
model = DecisionLM.from_pretrained("OVHaiLLM/Olala-v1.0-mini-7A1B").to("cuda").eval()
model.load(path)
model.systemone({"state": "...", "questions": {...}})
Read more
DetailsBenchmarks: 169 out of 231 on public JevBench, 72.1% on generated dev, 85.2% in-domain, and 61.2% on held-out tasks (PIQA, ANLI, WinoGrande); ECE on complex tasks is 0.142 (with threshold) and 0.112 (without threshold).
Hardware and speed: requires loading all 6.8B parameters (~1B active per token); processing a decision on a single H200 takes ~0.18 s, text generation takes ~12 tokens/s.
Limitations: the model does not reach the target calibration threshold of 0.10 in any configuration; text inside
state should be considered untrusted due to injection risks.Training: 2 stages (4,938 and 920 steps) on an H200, calibrator trained via PCA 32 with L2 regularization of 0.001.










Comments