MiniMax-H3-Prompt-Rewriter: Quantized Version of the Multimodal Model
2 min readupdated
The model is a quantized (compressed) version of lightx2v/MiniMax-H3-Prompt-Rewriter-Qwen3.8-27B-Ref2VA created using the llm-compressor library. The primary task of this LLM (large language model) is enhancing and rewriting prompts (text queries) for generative pipelines, taking visual data into account. Thanks to 4-bit quantization, the model weight has dropped from approximately 55 GB to 19 GB in VRAM, allowing it to run on consumer hardware and within ComfyUI environments with extra memory headroom. The model was created by developer 4kun and is geared toward enthusiasts and users of local AI stations.
Key features
- Heavy compression: model weight reduced from ~55 GB to ~19 GB VRAM in vLLM.
- NVFP4A16 quantization scheme (4-bit FP4 weights, 16-bit BF16 activations).
- Targeted optimization: only the linear layers of the language backend are quantized, while the vision module and lm_head are kept in original bfloat16.
- Convenient integration for running in environments like ComfyUI or vLLM servers.
What you can do
- Run the model locally on hardware with lower video memory requirements thanks to the compressed format.
- Connect the model in ComfyUI for automated enhancement and detailing of prompts for content generation.
- Set up a serving server via vLLM for fast multimodal request processing.
Example
An example of launching the model service via vLLM with support for image processing and generation configuration:
vllm serve 4kun/MiniMax-H3-Prompt-Rewriter-Qwen3.8-27B-Ref2VA-NVFP4A16 \
--served-model-name MiniMax-H3-Prompt-Rewriter-Qwen3.8-27B-Ref2VA \
--dtype bfloat16 \
--max-model-len 32768 \
--limit-mm-per-prompt '{"image":50,"video":0}' \
--reasoning-parser qwen3 \
--default-chat-template-kwargs '{"enable_thinking":true,"reasoning_effort":"xhigh"}' \
--override-generation-config '{"temperature":0.7,"top_p":0.8,"top_k":20,"max_new_tokens":16384}'Read more
DetailsQuantization scheme: NVFP4A16 (nvfp4-pack-quantized, W4A16)
VRAM requirements: ~18.8 GB in vLLM (versus ~55.6 GB in original BF16)
Architecture features: language backend quantized, vision tower and lm_head left in bfloat16
Base model: lightx2v/MiniMax-H3-Prompt-Rewriter-Qwen3.8-27B-Ref2VA














Comments