camera-island-lora
identifying rare animals and species on camera traps
2 min readupdated
This model tackles two specific tasks: classifying field camera trap images and recognizing endemic Galapagos Islands species that standard tools like SpeciesNet lack labels for. It is built on the lightweight LLaVA-OneVision 0.5B multimodal model (around 0.89B total parameters), fine-tuned with LoRA (a small trainable adapter added on top of frozen weights) on 2,902 labeled frames. It outperforms the base model and standard tools on specialized tasks like telling iguanas apart or spotting invasive rats. The project is geared toward ecologists, wildlife researchers, and developers building conservation systems.
Key features
- Specialized focus: trained to recognize rare Galapagos species (iguanas, mockingbirds) as well as pests.
- Lightweight architecture: based on the compact LLaVA-OneVision (0.5B parameters), keeping deployment simple.
- High accuracy: scores up to 93.3% on tough benchmarks distinguishing similar birds and reptiles.
- Fast training: the LoRA adapter was trained in just three hours on a single desktop machine (Mac Studio).
What you can do
- Camera trap analysis: automatically identify animals in field camera footage against a predefined species list.
- Wildlife conservation: classify endemic Galapagos animals and detect invasive rats inside nature reserves.
- Ecological monitoring: plug the model into field research pipelines to sort image batches quickly.
- Custom pipelines: use the repository as a starting point to build your own lightweight multimodal classifiers.
Example
import torch
from PIL import Image
from transformers import AutoProcessor, LlavaOnevisionForConditionalGeneration
repo = "andrewxhill/camera-island-lora"
processor = AutoProcessor.from_pretrained(repo)
model = LlavaOnevisionForConditionalGeneration.from_pretrained(repo, dtype=torch.float32).eval()
question = "Which animal is shown in this crop?\nA. marine iguana\nB. common green iguana"
messages = [{"role": "user", "content": [{"type": "image", "image": Image.open("crop.jpg").convert("RGB")}, {"type": "text", "text": question}]}]
inputs = processor.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt")
out = model.generate(**inputs, do_sample=False, max_new_tokens=64)
print(processor.decode(out[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))Read more
DetailsModel accuracy: 93.33% on birds (mockingbird vs. brown noddy) and 91.69% on marine iguanas.
Training setup: LoRA rank 8 (alpha 16), targeting query and value matrices across all 24 language blocks (540,672 trainable parameters total).
License: CC BY-NC 4.0 (base components use Apache-2.0; iNaturalist dataset carries non-commercial restrictions).










Comments