ColBERT Atto 90k
A Compact Model for Semantic Search
2 min readupdated
ColBERT Atto 90k is an ultra-compact model for semantic search and text representation tasks, built on the Atto architecture. Trained using the PyLate library, the model operates on the ColBERT principle (late interaction model), enabling it to efficiently match queries and documents at the token level. The model contains 90 thousand parameters, has a vector dimension of 32, and features a vocabulary of 4 thousand tokens. It is exceptionally well-suited for building embedding databases, RAG (Retrieval-Augmented Generation) systems, and serving as a reranking pipeline under resource-constrained conditions.
Key features
- Compact size: only 90K parameters with a dimension of 32.
- Support for late interaction for more accurate search.
- Integration with the txtai library (version 9.0+) and Sentence Transformers (version 6.0.0+).
- Average NDCG@10 score on BEIR benchmarks is 0.4454.
What you can do
- Build an embedding database for semantic search using txtai.
- Use the model as a knowledge source for RAG systems.
- Set up a reranker pipeline to refine search results.
- Integrate the model into applications via MultiVectorEncoder in Sentence Transformers.
Example
Running the model for feature extraction and relevance scoring via Sentence Transformers:
from sentence_transformers import MultiVectorEncoder
model = MultiVectorEncoder("neuml/colbert-atto-90k-32d")
query = "What is the capital of France?"
documents = [
"Paris is the capital and largest city of France.",
"Berlin is the capital of Germany.",
]
query_embeddings = model.encode_query(query)
document_embeddings = model.encode_document(documents)
scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
Read more
DetailsBEIR benchmarks (ndcg@10): ArguAna — 0.3435, NFCorpus — 0.3263, SciFact — 0.6663, average — 0.4454.
Training parameters: batch size 32, 1 training epoch, learning rate 0.01, warmup 0.05, fp16 support.
Requirements: supported by txtai 9.0+ and Sentence Transformers v6.0.0+.










Comments