DFM-Mimir-v1.8-Mini
a compact multilingual model on the HRM-Text architecture
2 min readupdated
DFM-Mimir-v1.8-Mini is a compact 703.6M-parameter text generation model from Danish Foundation Models. Unlike standard transformers, it uses the HRM-Text architecture, combining hierarchical recurrent computation with a PrefixLM training objective (a mode that blends bidirectional prefix context with autoregressive generation). Trained on the DFM15 (MTC v1.8) data mix—including DBC and Lex.dk materials—and evaluated across 50 languages, it's a solid fit for researchers exploring non-standard recurrent architectures and developers looking for a lightweight multilingual model under a permissive Apache-2.0 license.
Key features
- HRM-Text architecture: configured with H=2 and L=3 parameters, the full 262,144-token Gemma tokenizer, and thinking mode disabled.
- Context and weights: features a base context length of 4,096 tokens (extended to 8,192 via parameter overrides during evaluation), with weights published in BF16 (final epoch 3 EMA checkpoint at step 1,538,766).
- 50-language evaluation: benchmarked across 396 metrics in 50 languages, with included zero-shot and few-shot EuroEval results.
- Custom runtime required: needs an environment with HRM and PrefixLM support (tested with modified builds of Transformers 5.17.0 and vLLM 0.31.0).
What you can do
- Explore and test HRM-Text: spin up a local OpenAI-compatible API server via vLLM to experiment with hierarchical recurrent generation.
- Handle multilingual text tasks: generate and process text across European and other languages from its 50-language evaluation suite, including Danish and English.
- Run comparative benchmarks: reproduce EuroEval protocols or cross-check model behavior against the included
evaluation_results.jsonreports.
Example
python -m vllm.entrypoints.openai.api_server \
--model ./DFM-Mimir-v1.8-Mini \
--tokenizer ./DFM-Mimir-v1.8-Mini \
--served-model-name danish-foundation-models/DFM-Mimir-v1.8-Mini \
--chat-template ./DFM-Mimir-v1.8-Mini/chat_template.jinja \
--host 127.0.0.1 --port 8200 \
--attention-backend FLASH_ATTN --max-model-len 8192Read more
DetailsAverage score across 396 metrics in 50 languages is 66.0075 (66.3236 excluding English and Danish), outperforming the step 1,500,000 checkpoint on average score in 33 out of 50 languages.
Standard Transformers and vLLM builds without HRM support will not work, and the repository does not provide remote-code auto-loading.
Keep
fix_mistral_regex=false at launch to avoid breaking tokenization.Model weights are licensed under Apache-2.0 (separate Gemma tokenizer terms apply).










Comments