Trout
A diffusion language model for pixel art
2 min readupdated
Trout is a compact 47.6M-parameter model that generates 16×16 pixel sprites using masked diffusion. The model processes each sprite as a sequence of 256 tokens over a 512-color palette without using a VAE or latent space. The architecture is built on a bidirectional transformer with an hourglass structure and a CLIP text encoder. This release will interest developers and researchers exploring alternative image generation approaches without heavy diffusion pipelines.
Key features
- 512-color palette: uses a fixed OKLab k-means palette for pixel tokenization
- Parallel generation: unmasks all 256 pixels in about 16 steps using a confidence-ordered sampler
- Trained from scratch: entirely independent training process based on the Tessera-small recipe with the Muon optimizer
- Easy deployment: the model is distributed as a single fp16 weight file with an open PyTorch implementation script
What you can do
- Pixel art generation: create 16×16 sprites from text descriptions (such as game icons or characters)
- Sprite inpainting: complete or modify specific parts of existing images using masks
- Architecture exploration: study how masked diffusion works on discrete data without heavy weights
Example
from huggingface_hub import hf_hub_download
from trout import load_model_from_checkpoint, FrozenTextEncoder, sample, tokens_to_images, make_grid
ckpt = hf_hub_download("FrameXlabs/Trout", "trout-fp16.pt")
model, palette, meta = load_model_from_checkpoint(ckpt, device="cpu")
enc = FrozenTextEncoder(meta["text_encoder"], model.cfg.text_len, "cpu")
states, pooled = enc.encode(["golden crown with blue gems"] * 4)
toks = sample(model, states, pooled, steps=16, cfg_scale=3.0)
images = tokens_to_images(toks, palette)
make_grid(images, ncols=4).save("crowns.png")Read more
DetailsParameters: 47.6M
Metrics: validation NLL 2.176 nats/pixel, accuracy 54.6%
Requirements: runs and tested on CPU, weights in fp16 format
License: Apache-2.0










Comments