LocateAnything-3B
Accurate text-prompted object detection in images
2 min readupdated
This object-detection model locates specific elements in an image based on a text query. It takes an image and a text instruction, then returns the coordinates of the detected regions in a structured format. Based on a replication of a model from nvidia, it runs on the transformers library and has around 3.83 billion parameters. The tool is useful for developers and researchers who need to automatically find objects, faces, or specific details in visual media using text prompts.
Key features
- Full compatibility with modern versions of the transformers library
- Support for text prompts to locate arbitrary objects in an image
- Structured coordinate output for detected regions
- Image processing using the standard processor and model from the Hugging Face ecosystem
What you can do
- Integrate the model into computer vision pipelines for labeling and object search
- Automatically find specific details or parts in user-uploaded photos
- Use it as a foundation for visual assistants and multimodal systems
Example
import argparse
from pathlib import Path
import torch
from PIL import Image
from transformers import AutoModel, AutoProcessor
repo_id = 'SD-Next/LocateAnything-3B'
processor = AutoProcessor.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModel.from_pretrained(repo_id, trust_remote_code=True, torch_dtype="auto", device_map="auto").eval()
image = Image.open("image.png").convert("RGB")
messages = [{
"role": "user",
"content": [
{"type": "image", "image": image},
{"type": "text", "text": "locate hands and face"}
]
}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(images=image, text=text, return_tensors="pt").to(model.device)
with torch.inference_mode():
output = model.generate(**inputs, max_new_tokens=128, do_sample=False)
decoded = processor.batch_decode(output[:, inputs["input_ids"].shape[1]:], skip_special_tokens=False)
print(decoded[0])
Read more
DetailsTask: object-detection
Base model: nvidia/LocateAnything-3B
Parameters: 3.83B
Library: transformers










Comments