gemma4-e2b-it-ConvFill-TR
Turkish model for low-latency voice agents
2 min readupdated
A specialized Turkish fine-tune of google/gemma-4-E2B-it trained for Conversational infill. It acts as a fast local "Talker" agent, instantly responding to the user with short utterances and conversational fillers while a larger cloud model retrieves information in parallel. This architecture cuts latency in voice assistants without sacrificing heavy backend reasoning. It is built for developers creating interactive voice bots and hybrid local-remote assistant pipelines.
Key features
- Tailored for Turkish dialogue routing and response generation.
- Implements the Conversational infill approach from the ConvFill paper to minimize voice agent latency.
- Built on
google/gemma-4-E2B-itand trained on a synthetic dataset of 10,000 Turkish dialogues. - Available in ONNX (q4f16, WebGPU) and GGUF formats for llama.cpp.
What you can do
- Integrate the model as the local "Talker" module in a Turkish voice assistant.
- Pair it with a cloud backend that streams knowledge into a dedicated host context mid-conversation.
- Run the ONNX or GGUF builds on a consumer GPU to benchmark real-time response speeds.
Example
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "atasoglu/gemma4-e2b-it-ConvFill-TR"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto")
prompt = (
"<|turn>system\nSen Türkçe konu&an sesli bir asistansın.\n"
"<|turn>user\nYarın İstanbul'da hava nasıl olacak?\n"
"<|turn>model\n"
)
ids = tok(prompt, add_special_tokens=False, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=160, eos_token_id=[1, 106, 50], do_sample=False)
print(tok.decode(out[0, ids.input_ids.shape[1]:]))
Read more
DetailsValidation bpb (bits per byte) metrics: overall — 0.3661, filler/pause tokens — 0.4642, backend-grounded responses — 0.3317.
Training: 4-bit QLoRA via Unsloth (r=64, alpha=64), 1 epoch, peaking at roughly 8.2 GB of VRAM.
Limitations: Turkish only; factual knowledge must be supplied by an external backend model rather than the local agent.










Comments