OmniCore-7B-Merged: a multitask LLM based on Qwen2.5
1 min readupdated
OmniCore-7B-Merged is a 7.6-billion-parameter large language model (LLM) built by merging multiple Qwen2.5 variants using TIES (a technique for combining neural network weights). The merge pulls together base, instruct, math, and coding versions of the model alongside a specialized SFT checkpoint (a model fine-tuned on instructions). As a result, it balances programming, STEM reasoning, and everyday conversation—making it a solid all-around workhorse for developers and researchers handling a wide range of text tasks.
Key features
- Built on the Qwen2.5 family, combining capabilities across multiple domains.
- Merged with TIES using normalization and int8 masks for cleaner weight blending.
- Incorporates specialized variants for coding, math, and general instruction following.
- Supports bfloat16 precision for efficient inference.
What you can do
- Tackle complex programming problems and write code across multiple languages.
- Solve math problems and analyze documents containing calculations.
- Use it as a general-purpose assistant for generating and processing text.
Example
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "adityaxess/OmniCore-7B-Unified"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
Read more
DetailsMerge method: TIES (with normalization and int8_mask)
Base model: Qwen/Qwen2.5-7B
Precision: bfloat16
Parameter count: 7.62B










Comments