PaleVision: An evolutionary approach to spectral vision
2 min read
A new research model from NeurIPS 2026 explores the evolution of color vision (EVPL — evolutionary visual perception layers) for semantic segmentation tasks on hyperspectral images. The solution models the spectral responses of opsins and combines them with convolutional networks and segmentation heads to handle complex spectral data. The development targets computer vision and spectral data processing researchers who need to reproduce experiments or use pre-trained weights to analyze specific datasets.
Key features
- EVPL architecture: The use of trainable spectral response layers based on biological models of vision evolution.
- High metrics: The availability of pre-trained state-of-the-art weights for the MODID, MHSI, and MinnApple datasets with fixed mean intersection over union results.
- Configuration flexibility: Support for various backends, including MiT-B0, Twins-SVT-Small, and PVT-v2-B2.
- Training tools: PyTorch and torchrun scripts for multi-process experiment execution.
What you can do
- Mars analysis: Run training and evaluation of spectral filters and segmentation on Mars MSL data.
- Agricultural data processing: Apply checkpoints for detection, counting, and segmentation on the MinnApple dataset.
- Encoder testing: Perform encoder swaps (encoder_swap_exp.py) with customizable opsin modes.
- Weight inference: Load pre-trained checkpoints (backbone, head, and EVPL) in Python scripts for testing on custom data.
Example
import torch
ckpt = torch.load("weights/modid_twins_svt_small_free.pth", map_location="cpu")
backbone.load_state_dict(ckpt["backbone"])
head.load_state_dict(ckpt["head"])
evpl.load_state_dict(ckpt["evpl"])Read more
DetailsBenchmarks: MODID (mean intersection over union 0.6006), MHSI (mean intersection over union 0.5727), MinnApple segmentation (mean intersection over union 0.4845), MinnApple counting (mean absolute error 2.446).
Hardware: An NVIDIA graphics processing unit with sufficient memory for backends at config resolutions; Central Processing Unit loading is supported.
Limitations: Weights contain only module parameters without optimizer states; models are tightly coupled to their training datasets.










Comments