Sloth-27B-v1
26B model for routing and decision-making
2 min readupdated
Sloth-27B-v1 by cjqhhh is a specialized model for making structured decisions from text or JSON context. Instead of generating free-form text, it picks one of the predefined options and returns a probability distribution across all candidates. Built on vllm-sr/d3, the model packs around 26.09B parameters and features a dedicated FP32 classification head (decision head). It is built for developers tackling query routing, relevance scoring, and business-rule automation without risking output formatting hallucinations.
Key features
- No free-form generation: skips extra text and strictly outputs a typed decision (selecting from up to 255 options or returning a binary answer) alongside probabilities.
- Batch question handling: lets you ask multiple questions against a single shared state (context) in one batch.
- Flexible input format: accepts context as plain text or a JSON object paired with instructions and criteria for each option.
- Text-only focus: vision weights inherited from the base model were not tested for this release.
What you can do
- Support ticket routing: automatically dispatch incoming tickets and user messages across departments (billing, tech support, sales) based on set criteria.
- Filtering and relevance scoring: verify whether documents or responses match specific rules via binary checks (
noul) with truth-probability scores. - AI agent orchestration: deploy the model as a fast, strict router to pick the next step in multi-stage data processing workflows.
Example
from transformers import AutoModel
model = AutoModel.from_pretrained("./Sloth-27B-v1", trust_remote_code=True, device="cuda:0")
res = model.system_one(
state={"message": "Please send me the invoice for my order."},
questions={"route": {"type": "choice", "instructions": "Choose team", "criteria": {"billing": "Invoices", "sales": "Enquiries"}}}
)
print(res["answers"]["route"]["choice"])
Read more
DetailsParameters: ~26.09B (11 BF16 safetensors shards) + FP32 readout head for 255 response codes.
Requirements: weights take up around 52.2 GB; requires Linux with a CUDA-capable GPU.
Limitations: probability calibration has not been independently evaluated; GPU inference was not re-tested for this release.










Comments