JANSORI
a 3.22B bilingual base model
2 min readupdated
JANSORI is a pretrained 3.22-billion-parameter base LLM with a dense architecture created by pathcosmos. It is designed for text completion in Korean and English without chat templates or role tokens. Unlike instruct-tuned models, it is a raw base model intended as a starting point for fine-tuning on custom tasks. It suits researchers and developers working with Korean-English text or studying the behavior of compact models prior to SFT.
Key features
- Bilingual focus: trained on roughly 98.57 billion tokens split nearly evenly between Korean (Wikipedia, FineWeb-2, HPLT 2.0) and English data, including math (OpenWebMath) and code (GitHub Code).
- Ready-to-use quantizations in the repo: alongside the original BF16 weights,
Q8_0andQ4_K_MGGUF builds with SHA-256 checksums are included in the same commit. - Compact context and vocabulary: supports a context length of up to 4,096 tokens and uses a 65,536-token vocabulary.
- Transparent breakdown of limitations: the author provides detailed perplexity (PPL) benchmarks, generation speeds, and failure examples across logic, arithmetic, and code.
What you can do
- Use it as a fine-tuning base: adapt the model via LoRA or full SFT for Korean and English text generation or processing.
- Run locally with Ollama or vLLM: test text completion on a consumer GPU using GGUF builds, or deploy a server in BF16.
Example
ollama run hf.co/pathcosmos/JANSORI:Q8_0 "한국의 수도는"
ollama run hf.co/pathcosmos/JANSORI:Q4_K_M "한국의 수도는"
Read more
DetailsHoldout metrics: Korean PPL — 9.09 (BF16 / Q8_0) and 9.20 (Q4_K_M); English PPL — 10.22 (BF16 / Q8_0) and 10.32 (Q4_K_M).
VRAM usage during generation (measured on an H200): 8.8 GB for BF16, 5.06 GB for Q8_0, and 3.76 GB for Q4_K_M.
Training ran for 47,000 steps (98,566,144,000 tokens).
Limitations: like other raw base models without fine-tuning, it tends to repeat phrases, struggles with basic arithmetic, and drops indentation in code.










Comments