Sample-Aligned Decoders (SAD)
drop-in decoders for image generators
2 min readupdated
A collection of pretrained decoders from researchers at the Chinese Academy of Sciences and Tsinghua University that replace the default autoencoder decoder (VAE — the module that turns a compressed image representation into pixels) in latent generative models. Each file contains only the decoder weights (decoder.* and post_quant_conv.*), fine-tuned with Sample-Aligned Decoders on the latents of a specific generator. The encoder, latent space, and generator itself remain unchanged, keeping inference speed and cost identical. Built for researchers and developers working with DiT, SiT, and PixArt-α.
Key features
- Act as a drop-in replacement — swap the decoder in place without retraining the base generator.
- Add zero inference overhead — the architecture and compute cost stay identical to the original.
- Support popular generator families: DiT-XL/2, SiT-XL/2 + REPA, LightningDiT-XL, REPA-E, and PixArt-α (at 256px resolution).
- Include repository variants trained via GAN discrepancy in DINOv2 feature space (
sad-gan) and via MMD (sad-mmd).
What you can do
- Swap the decoder in
diffuserspipelines for DiT-XL/2 or PixArt-α 256px models using theload_sad_decoderhelper or standardload_state_dict. - Load the
lightningdit-xl-vavae-256andrepa-e-e2evae-256weights in the LightningDiT and REPA-E codebases for VA-VAE and E2E-VAE autoencoders. - Train your own SAD decoder tailored to your specific generator and sampler following the instructions in the project repository.
Example
from diffusers import DiTPipeline
from sad import load_sad_decoder
pipe = DiTPipeline.from_pretrained("facebook/DiT-XL-2-256").to("cuda")
load_sad_decoder(pipe.vae, "dit-xl2-256/sad-gan")
images = pipe(class_labels=[279, 388], num_inference_steps=250, guidance_scale=1.5).images
Read more
DetailsTraining: each decoder was fine-tuned for 2,000 steps (~1 GPU-hour) using a GAN loss in DINOv2 space or MMD, combined with an L1 encoder anchor.
Limitations: each checkpoint is tied to its specific generator, tokenizer (SD-VAE ft-EMA, VA-VAE, E2E-VAE), and sampler.
Compatibility: weights for VA-VAE and E2E-VAE are not directly compatible with
diffusers and require classes from LightningDiT and REPA-E.









Comments