LFM3-VL-2B
a compact multimodal model with video and reasoning support
2 min readupdated
Liquid AI's new compact multimodal model (1.92B parameters) is built for on-device visual understanding. It takes text, images, and—for the first time in the LFM lineup—video inputs, and uses a built-in thinking block to reason step by step before answering. Built on the LFM3-1.7B-Base language model and a new proprietary vision encoder, it supports a context window of up to 128K tokens. It excels at chart and document analysis as well as spatial object positioning, making it a solid fit for developers and hobbyists building local multimodal apps and robotics.
Key features
- Video understanding: the first LFM-series model capable of processing video clips, defaulting to 2 fps.
- Step-by-step reasoning: thinks before answering, which helps with complex visual tasks, math, and chart analysis.
- Spatial reasoning: detects objects in a frame and analyzes their relative positions for robotics tasks.
- Multilingual support: handles 16 languages, including Russian.
What you can do
- Video analysis: ask questions about recorded lectures, phone clips, or security camera footage.
- Chart and document reading: extract text and data from scanned documents, app interfaces, and charts.
- Robotics and edge devices: use spatial reasoning for local object detection on robots and smart gadgets.
- Local deployment: run the model locally via llama.cpp (GGUF), vLLM, or SGLang for fast inference.
Example
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
model_id = "LiquidAI/LFM3-VL-2B"
model = AutoModelForImageTextToText.from_pretrained(
model_id, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True
)
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
messages = [
{
"role": "user",
"content": [
{"type": "image", "url": "[https://placecats.com/300/200](https://placecats.com/300/200)"},
{"type": "text", "text": "Describe this image."},
],
}
]
inputs = processor.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.inference_mode():
output_ids = model.generate(
**inputs,
do_sample=True,
temperature=0.1,
top_k=50,
repetition_penalty=1.1,
max_new_tokens=16384,
)
generated_ids = output_ids[:, inputs["input_ids"].shape[1] :]
print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0])
Read more
DetailsParameters: 1.92B (1.73B language model, 164M vision encoder, 25M projector).
Context: 128K tokens.
Architecture: 24 language model layers (combining convolutions and attention) and 14 vision encoder layers.
Limitations: not recommended for coding, multi-turn conversations, or complex reasoning. License: LFM Open License v1.0.










Comments