One-For-All-4B-GGUF
quantized model files by mradermacher
2 min read
This repository provides static quantizations (GGUF builds) for the multimodal model TendieLabs/One-For-All-4B. The GGUF format lets you run language and multimodal models on consumer hardware (such as a standard PC or laptop using llama.cpp and compatible tools). Available compression options range from fast, compact files to higher-precision builds (including Q4_K_M, Q8_0, IQ4_XS, and others), alongside files for the multimodal projector (mmproj). It is a great fit for developers and hobbyists looking to run this architecture locally with minimal VRAM requirements.
Key features
- Full suite of static GGUF quantizations (including Q2_K, Q3_K_M, Q4_K_M, Q8_0, and more)
- Multimodal projector companion files included (mmproj-f16 and mmproj-Q8_0)
- Optimized for local inference on standard consumer hardware
- Multiple compression levels to balance output quality and file size
What you can do
- Run the model locally using compatible clients and libraries (such as llama.cpp)
- Pick the right quantization size to fit your available RAM or VRAM
- Integrate multimodal capabilities into local data processing pipelines
Example
Using these GGUF files follows the standard llama.cpp workflow. Point your preferred client or CLI utility to the downloaded model file and its matching mmproj file to get started.
Read more
DetailsBase model: TendieLabs/One-For-All-4B
Library: transformers / GGUF
Available formats: x-f16, Q4_K_S, Q2_K, Q8_0, Q6_K, Q3_K_M, Q3_K_S, Q3_K_L, Q4_K_M, Q5_K_S, Q5_K_M, IQ4_XS
Multimodal components: mmproj-Q8_0 (0.5 GB), mmproj-f16 (0.8 GB)










Comments