tiny-embed-27m
a compact and efficient Chinese text embedding model
2 min readupdated
The tiny-embed-27m model converts Chinese text into dense vector representations suited for semantic search, classification, clustering, and retrieval. Distilled from larger open models (including the BAAI bge family), it packs 27.6 million parameters with an output dimension of 512. Its standout architectural feature is recursive depth, where 4 unique layers are applied 16 times. The model needs no special query prefixes and outperforms similarly sized alternatives in benchmarks.
Key features
- High performance: Scores an average of 0.5963 on the MTEB benchmark (Chinese suite, 31 tasks), outperforming bge-small-zh-v1.5 (24M).
- Recursive architecture: 4 unique layers repeated 16 times keep the parameter count low while maintaining high inference speed.
- Ready-to-use pipeline: Uses a standard
Transformer → Pooling(mean) → Normalizesetup with no extra projection layers. - No prefixes needed: Queries and documents take the exact same raw text without additional instructions.
What you can do
- Semantic search: Plug the model into search engines to match short queries against long Chinese text documents.
- Clustering and classification: Use vector representations to group similar texts and documents.
- Quick integration: Run the model via the popular
sentence-transformerslibrary withtrust_remote_code=Trueenabled.
Example
from sentence_transformers import SentenceTransformer
sentences_1 = ["样例数据-1", "样例数据-2"]
sentences_2 = ["样例数据-3", "样例数据-4"]
model = SentenceTransformer("dynasty-ai/tiny-embed-27m", trust_remote_code=True)
embeddings_1 = model.encode(sentences_1, normalize_embeddings=True)
embeddings_2 = model.encode(sentences_2, normalize_embeddings=True)
similarity = embeddings_1 @ embeddings_2.T
print(similarity)
Read more
Details• Metrics: Average MTEB(cmn, v1) score is 59.63 (compared to 57.34 for bge-small-zh-v1.5).
• Limitations: Maximum sequence length is 512 tokens.
• License: Released under the Apache-2.0 license, allowing commercial use.
• Tokenizer: Uses the standard Chinese BERT vocabulary of 21,128 tokens.










Comments