Skip to content
Other

Metis Hybrid Olala mini

a hybrid model for structured prompts

2 min read

This model is a LoRA adapter (a small add-on over a base model) and a calibrator built on top of the frozen base OVHaiLLM/Olala-v1.0-mini-7A1B (6.8B total parameters, roughly 1B active per token) by GreenPT. It is designed to handle TypeSafe Jev format prompts via /v1/systemone with calibrated probabilities while preserving standard chat features. Unlike the standard base, it features an added decision head and frame-by-frame calibrator.pt calibrator, and uses a shortened system prompt for requests. The model is useful for developers who need structured solutions with numerical confidence scoring.

Key features

  • Hybrid architecture: combines a LoRA adapter (rank 16), three trained rows of special tokens, a decision head, and a calibrator.pt calibrator.
  • Probability calibration: outputs calibrated confidence scores for answers to prompts, reducing the discrepancy between confidence and actual accuracy.
  • Three operation modes: supports answer-only output (systemone), a combined mode with explanations (both), and a standard language model mode (text).
  • Ready-to-use wrapper: includes the integrated_decisions_v2 loading code and the scripts/serve.py server startup script.

What you can do

  • Deploy an API server: spin up a local service for processing POST requests to /v1/systemone via the serve.py script inside the OVHai LLM Docker image.
  • Integrate into Python: connect to the model using the DecisionLM class for programmatic processing of states and questions.
  • Get decision explanations: run in mode="both" to receive textual justifications for each answer alongside the probabilities.
  • Use in chat mode: conduct a regular dialogue using the standard system prompt of the Olala mini base model.

Example

import os, sys
from huggingface_hub import snapshot_download

os.environ["IDV2_BACKBONE_REVISION"] = "0fde460f37d0fe2c390619bb5729c9d81dc89091"
path = snapshot_download("GreenPT/metis-hybrid-olala-v1.0-mini")
sys.path.insert(0, path)
from integrated_decisions_v2 import DecisionLM

model = DecisionLM.from_pretrained("OVHaiLLM/Olala-v1.0-mini-7A1B").to("cuda").eval()
model.load(path)
model.systemone({"state": "...", "questions": {...}})
Read moreDetails
Benchmarks: 169 out of 231 on public JevBench, 72.1% on generated dev, 85.2% in-domain, and 61.2% on held-out tasks (PIQA, ANLI, WinoGrande); ECE on complex tasks is 0.142 (with threshold) and 0.112 (without threshold).
Hardware and speed: requires loading all 6.8B parameters (~1B active per token); processing a decision on a single H200 takes ~0.18 s, text generation takes ~12 tokens/s.
Limitations: the model does not reach the target calibration threshold of 0.10 in any configuration; text inside state should be considered untrusted due to injection risks.
Training: 2 stages (4,938 and 920 steps) on an H200, calibrator trained via PCA 32 with L2 regularization of 0.001.
Discuss Telegram 0

Will it run?

Runs in BF16

Fits without quantisation.

  • BF16weights 13 GB · ready files14 GB
  • FP8 / INT8recommendedweights 6.4 GB7.5 GB
  • 4-bitweights 3.6 GB4.5 GB

Tip: go 8-bit.FP8 / INT8 is within about a percent of full precision in quality, takes half the memory and runs faster on RTX 40/50 and H100. Full precision only pays off for fine-tuning.

What to get for this model

Massed ComputeOpen Massed Compute

Your own GPU: FP8 / INT8 — from RTX 5060 Ti 8GB · BF16 — from RTX 5080

Massed Compute hourly prices, November 2026. The discounted price applies with the coupon at checkout.

This is a LoRA: counted together with its base model Olala-v1.0-mini-7A1B.

Estimate: weights plus working memory. Long videos, high resolutions and long contexts need more.

Run it on Massed Compute

The script downloads the model to the rented machine. How to run it is in the model card.

  1. 1. Rent a machine with a fitting GPU

    Open Massed Compute
  2. 2. Paste the script into the machine's terminal

    Connect over SSH (the Massed Compute panel shows the command) or open a terminal on the machine's desktop.

    HF_TOKEN / CIVITAI_TOKENThe script starts with token lines: paste yours between the quotes. The Hugging Face token is for gated models, the Civitai one for Civitai models.

    # GreenPT/metis-hybrid-olala-v1.0-mini
    # ===== Tokens: paste yours between the quotes =====
    export HF_TOKEN=""        # huggingface.co/settings/tokens, for gated models
    # ==========================================
    mkdir -p ~/models
    curl -LsSf https://astral.sh/uv/install.sh | sh
    source "$HOME/.local/bin/env"
    uv tool install "huggingface_hub[cli]"
    hf download OVHaiLLM/Olala-v1.0-mini-7A1B --local-dir ~/models/Olala-v1.0-mini-7A1B
    hf download GreenPT/metis-hybrid-olala-v1.0-mini --local-dir ~/models/metis-hybrid-olala-v1.0-mini
    # The model is in ~/models. How to run it is in the model card (link above).
    Add more LoRAs to Olala-v1.0-mini-7A1B
  3. 3. Use it

    The model is in ~/models. How to run it is in the model card (link above).

Stop the machine in the panel when you are done: it bills by the minute while it runs.

How to speed it up

Picked for this model type. Choose your hardware above to see what fits you exactly.

  • FlashAttention 44.0.0b40from November 18, 2026new release

    fastest exact attention on B200

    Kernels for Blackwell (beta). Exact attention without quantisation — fine for everything, LLMs included.

    How to enable: pip install flash-attn-4; vLLM and SGLang pick it up themselves.

    Earlier generations (2)
    • FlashAttention 3 — The Hopper (H100/H200) version with FP8.
    • FlashAttention 2 — The standard for RTX 30/40/50 and A100: exact attention without the memory cost of long contexts.
  • torch.compile2.15.0from October 28, 2026new release

    10–30% for free

    Compiles the model for your GPU. Slower first run, faster afterwards.

    How to enable: model = torch.compile(model); in ComfyUI the TorchCompileModel node.

Model family

Requires the base model

OVHaiLLM/Olala-v1.0-mini-7A1B

This model is a LoRA / adapter. Download the base model first, then load this one on top.

  1. Olala-v1.0-mini-7A1B-basefamily root
  2. Olala-v1.0-mini-7A1B-SFT
  3. Olala-v1.0-mini-7A1B
  4. metis-hybrid-olala-v1.0-mini

New posts about versions of Olala-v1.0-mini-7A1B-base (quants, finetunes, LoRAs) will show up under the bell.

Fine-tunes 1All on Hugging Face

About the model

GreenPT/metis-hybrid-olala-v1.0-mini
Author
GreenPT
Category
Other
Released
Posted

On NeyroLearn

Hugging Face

00

Ask AI about this model

Answered by Gemini 4 · the chat is kept only in this tab

Comments