100 task experts clip-vit-l14: Computer vision expert models
2 min readupdated
The repository features a collection of 100 specialized expert models fine-tuned from the openai/clip-vit-large-patch14 visual encoder. Each subfolder handles a distinct visual analysis and classification task, ranging from standard datasets like MNIST and CIFAR to medical scans and satellite data. Built on top of the renowned vision-text encoder CLIP, these models significantly outperform the base version across specialized domains. This set is ideal for researchers and developers looking for ready-to-use trained weights for benchmarking and transfer learning.
Key features
- 100 expert checkpoints for various computer vision tasks
- Substantial accuracy boost over the zero-shot base CLIP model
- Clear structure featuring weights in safetensors format, metrics, and classes for each task
- Fully compatible with the standard Hugging Face Transformers library
What you can do
- Run image classification on specialized datasets (medicine, satellite imagery, automotive)
- Explore model merging capabilities and knowledge transfer across different visual domains
- Use individual subfolders with weight files as ready components for computer vision pipelines
- Integrate the models into research projects using the standard
CLIPVisionModelclass
Example
Example of loading a model for the MNIST dataset from a specific repository subfolder:
from transformers import CLIPVisionModel
m = CLIPVisionModel.from_pretrained("jhnb45C/clip-vit-l14-100tasks", subfolder="mnist/vision_model")
Read more
DetailsAverage metrics across 100 tasks: zero-shot test 48.11%, fine-tuned validation 90.67%, fine-tuned test 89.79%
Training parameters: Adam optimizer, lr 1e-05, cosine scheduler, 4,000 steps, batch size 128, fp32 precision with TF32
Weights released strictly for research purposes; original datasets retain their respective licenses










Comments