Harrier-oss-v1-0.6b: A powerful multilingual embedding model from Microsoft
1 min readupdated
This model converts text into dense vector representations (embeddings) for search, classification, and data clustering. Built on a decoder-only architecture with 0.6B parameters, it supports a context length of up to 32,768 tokens. It is trained using knowledge distillation from larger models and delivers excellent benchmark performance. It is useful for developers building search engines, semantic search systems, and recommendation systems across multiple languages, including Russian.
Key features
- Huge context support up to 32,768 tokens for long documents
- Multilingual capability supporting Russian and dozens of other languages
- High performance on the Multilingual MTEB v2 benchmark
- Uses last-token pooling (extracting the embedding from the final token)
What you can do
- Set up semantic search over documents or a knowledge base in your project
- Implement text classification and data clustering
- Perform bitext mining and results reranking
Example
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("microsoft/harrier-oss-v1-0.6b", model_kwargs={"dtype": "auto"})
queries = ["how much protein should a female eat"]
documents = ["As a general guideline, the CDC's average requirement..."]
query_embeddings = model.encode(queries, prompt_name="web_search_query")
document_embeddings = model.encode(documents)
Read more
DetailsBenchmark: MTEB v2 Score 69.0 for the 0.6B version
Architecture: Decoder-only with last-token pooling and L2 normalization
Training: Contrastive learning and knowledge distillation










Comments