MiniMax-H3-Prompt-Rewriter-Qwen3.5-9B-Ref2VA-Q3_K_M-GGUF
a local prompt rewriter in GGUF format
2 min readupdated
This is a GGUF build (a format for running models locally on CPUs and GPUs) of lightx2v/MiniMax-H3-Prompt-Rewriter-Qwen3.5-9B-Ref2VA, converted by Bakanayatsu using llama.cpp. The original model is built for rewriting and enhancing prompts—judging by the name, for Ref2VA tasks in the MiniMax-H3 lineup based on the 9B Qwen3.5 model. Q3_K_M quantization (3-bit weight compression) significantly lowers memory requirements compared to the full-size checkpoint. It is a solid fit if you want to run a prompt generator and enhancer locally without heavy server hardware.
Key features
- Q3_K_M quantization — a compact 3-bit variant of the 9B model designed to run on consumer PCs.
- GGUF format with native support for
llama.cpp,llama-cli, andllama-server. - Direct loading from the Hugging Face repo via the
--hf-repoand--hf-fileflags with no manual download required. - Converted using the official
GGUF-my-repotool from ggml.ai.
What you can do
- Spin up a local API server with
llama-serverand plug it into generation pipelines as an automated prompt-enhancement step. - Run it in CLI mode via
llama-clito quickly refine text prompts offline.
Example
llama-cli --hf-repo Bakanayatsu/MiniMax-H3-Prompt-Rewriter-Qwen3.5-9B-Ref2VA-Q3_K_M-GGUF \
--hf-file minimax-h3-prompt-rewriter-qwen3.5-9b-ref2va-q3_k_m.gguf \
-p "The meaning to life and the universe is"Read more
DetailsBase model:
lightx2v/MiniMax-H3-Prompt-Rewriter-Qwen3.5-9B-Ref2VA.Format and quantization: GGUF, Q3_K_M (converted via llama.cpp / GGUF-my-repo).
Detailed architecture notes, benchmarks, and system prompts can be found on the original model card.














Comments