Jalebi-Loop-35M
a compact model for decision-making and routing
2 min read
Created by Airavat-ai, this model features just 35M parameters and is designed not for text generation, but for efficient option selection, classification, and criteria scoring. It takes context and a list of options as input, outputting scores for each choice in a single pass, which eliminates hallucinations and prompt injections. The architecture supports dynamic computation depth via a repeating middle block (ranging from K=1 for maximum speed to K=4 for accuracy). This solution is ideal for developers tackling query routing, filtering, and rapid custom fine-tuning on a single video card.
Key features
- Ultra-lightweight architecture: only 35M parameters, with VRAM consumption ranging from 0.3 to 0.6 GB.
- Flexible depth (K): switch between K=1 (high speed) and K=4 (enhanced accuracy) during inference.
- Three task types: choice processing, binary probabilities (noul), and criteria scoring (score).
- Fine-tuning ready: simple customization for custom business processes and classifiers.
What you can do
- Request routing: distribute incoming support tickets across departments (billing, fraud, support).
- Automatic classification: filter short, structured requests and texts without the risk of generating unnecessary junk.
- Option ranking: evaluate the relevance of pre-made options against specified criteria.
- Fast inference: deploy the model in services with strict latency requirements and minimal VRAM consumption.
Example
import sys, numpy as np
sys.path[:0] = ["code", "code/eval", "code/model"]
from s1mb_adapter import load_checkpoint
from heldout_eval import build_scorer
model, tok, meta = load_checkpoint("Jalebi-Loop.pt", "tok24k.json", device="cpu")
scorer = build_scorer(model, tok, 512, "cpu", layout=meta["layout"])
record = {
"case_id": "demo", "question_id": "demo", "task": "choice",
"state": "Customer writes: my card was declined while travelling abroad",
"question": "Where should this request go?",
"options": ["billing team", "fraud team", "general support"],
"option_ids": ["0", "1", "2"],
}
scores = scorer.score_records([record], 4)[0]["scores"][-1]
probs = np.exp(scores - scores.max()); probs /= probs.sum()
print(probs)Read more
DetailsMetrics: on the S1MB English v1 benchmark (K=4), the average task metric is 9.87.
Hardware: on an NVIDIA L4 in K=1 mode (bf16), the model delivers up to 411–421 requests per second with peak memory usage up to 0.38 GB.
Limitations: the context window is limited to 512 tokens, and each option is capped at 40 tokens. The model is not a text generator and performs poorly on long reasoning tasks.










Comments