qwen2.5-3b-toolcall-abstain-v6
accurate tool calling with abstention
2 min readupdated
This new language model, based on Qwen2.5-3B-Instruct, is fine-tuned for accurate tool calling and working with JSON schemas. Beyond generating valid function calls, it knows when to abstain if no suitable tool is available, or ask clarifying questions when required data is missing. Built for developers creating local AI agents and automation systems, the repository includes the merged model, a 4-bit version for Apple Silicon (via MLX), and a LoRA adapter.
Key features
- Built for agents: optimized for complex function-selection scenarios and filtering out irrelevant distractor tools.
- Reliable abstention: significantly outperforms the base model at recognizing when no tool call is needed.
- Mac-ready: includes a 4-bit
mlx/build to run efficiently on Apple Silicon via themlx-lmlibrary. - Efficient training: trained using QLoRA on public datasets (xLAM, Glaive, Hermes, ToolACE).
What you can do
- Build a local agent: connect the model to external APIs, databases, or tools through a custom pipeline.
- Automate tasks: use the model for structured function calling in bots and services.
- Run on Apple Silicon: deploy the 4-bit version locally on a Mac for fast inference.
Example
from transformers import AutoModelForCausalLM, AutoTokenizer
m = 'siva-0104/qwen2.5-3b-toolcall-abstain-v6'
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m, torch_dtype='auto')
Read more
DetailsBenchmarks: BFCL simple exact match — 89.7%, BFCL irrelevance abstained — 87.1%, xLAM exact — 77.3%.
Limitations: English only; when parameters are missing, the model occasionally guesses values instead of asking follow-up questions; retriever performance degrades when thousands of tools are available.
Training: QLoRA, 4-bit NF4, LoRA r=16 alpha=32, 1 epoch on a Kaggle T4 in 169 minutes.
License: inherits the Qwen Research License terms.










Comments