HiTZ/Latxa-3-122B-A10B
a multimodal language model for Basque
2 min readupdated
Latxa 3 is a large multimodal instruct model (trained to follow instructions) created by researchers at the HiTZ center and the IXA group (University of the Basque Country). Built on Qwen3.5-122B-A10B using a Mixture-of-Experts (MoE) architecture, it has 122 billion total parameters with roughly 10 billion active per token and supports context windows up to 262,144 tokens. Latxa 3 handles images and holds conversations in Basque, operating exclusively in thinking mode to reason through responses before answering. It was built to close the AI performance gap for low-resource languages.
Key features
- Support for image inputs and multimodal instructions.
- Mandatory thinking mode that reasons through answers before responding.
- Significant accuracy gains on Basque language and cultural knowledge benchmarks over the base model.
- Massive context window of up to 262,144 tokens.
What you can do
- Deploy a Russian or Basque AI assistant for enterprise or research workflows.
- Run multimodal image analysis with text prompts on top of a capable open-weights architecture.
- Integrate the model into services via an OpenAI-compatible API using frameworks like vLLM.
Example
Sample request passing an image to the model through an OpenAI-compatible client:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
messages = [
{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"url": "[https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG)"}},
{"type": "text", "text": "Zer ikusten da irudi honetan?"},
],
},
]
response = client.chat.completions.create(
model="Latxa-3",
messages=messages,
max_tokens=1024,
)
print(response.choices[0].message.content)Read more
DetailsBuilt on the Qwen3.5-122B-A10B architecture (MoE: 256 experts, 8 routed + 1 shared per token).
Scored an average of 84.0% on Basque language proficiency tasks and 76.4% on Basque knowledge benchmarks.
Hardware requirements: BF16 weights require roughly 245 GB of VRAM (e.g., 4x 80 GB GPUs). An FP8 version is also available.
License: Apache 2.0.










Comments