PKaI Nano 2 Instruct
a compact open-weights chat model
2 min readupdated
PKaI Nano 2 Instruct is the first instruction-tuned chat version of the base PKaI Nano 2 model by PowderKeg Intelligence. The model has about 0.84 billion parameters and was trained on a synthetic mix of multi-turn dialogues using a special pkai-chat-v1 format. The repository contains PKaI-native weights and metadata rather than a standard transformers package. It will be useful for developers and researchers who need a small local language model for chat tasks.
Key features
- Model size is 0.84B parameters with a 2048-token context length.
- Uses a specific
pkai-chat-v1chat format with a built-in template in the tokenizer. - Shows improved benchmark results compared to the previous first version of the Nano series.
- Distributed under the free Apache License, Version 2.0.
What you can do
- Integrate the model into local chat scripts and custom pipelines with support for PKaI native weights.
- Test text generation and question answering by passing custom system prompts.
- Explore the architectural features of compact models with QK-normalization and 30 layers.
Example
Running and working with the model uses the native weights and config format, as well as the built-in chat template from the tokenizer.json file. To set a custom context, pass an explicit system message, for example: "You are PKaI Nano 2 Instruct, a helpful assistant made by PowderKeg Intelligence.".
Read more
DetailsBenchmarks (base checkpoint): HellaSwag — 43.87%, SciQ — 85.40%, PIQA — 67.52%, ARC-Easy — 51.77%, 7-task average — 53.18%.
Architecture: 792,429,696 parameters, 30 layers, 16 attention heads, 4 KV heads, embedding size 1536.
Limitations: the model is small and has not undergone special safety tuning, so it may generate inaccurate or unwanted text.










Comments