SD15 no-VAE DWD
Image generation without VAE
2 min readupdated
This model is a modification of the well-known base model DreamShaper 8 LCM, adapted for direct pixel prediction using the DWD (Decoding Without a Decoder) method. The model predicts pixel patterns in the Oklab color space and converts them to RGB. The key feature is that text-to-image generation runs entirely without using a VAE (Variational Autoencoder, a component for encoding and decoding images). This tool will be useful for developers and enthusiasts experimenting with diffusion model architectures.
Key features
- Built on DreamShaper 8 LCM with training over 10,000 steps.
- Direct prediction of Oklab pixel patterns followed by RGB conversion.
- No need for a separate VAE module during inference.
- The export includes U-Net, a trained EMA tail, a pixel head, the CLIP text encoder, and an LCM scheduler.
What you can do
- Generate realistic landscapes, portraits, and still lifes from text descriptions without loading a VAE.
- Test the alternative DWD image decoding method in practice.
- Integrate the model into custom graphics generation pipelines built on Diffusers.
Example
pip install diffusers==0.35.2 transformers==4.57.6 accelerate==1.11.0
hf download ApacheOne/SD15_no_vae_dwd_trained dreamshaper_dwd.py dwd_inference.pt --local-dir ./sd15_dwd
python ./sd15_dwd/dreamshaper_dwd.py infer --checkpoint ./sd15_dwd/dwd_inference.pt --prompt "An old stone lighthouse by a stormy sea" --guidance 2 --seed 42 --output lighthouse.pngRead more
DetailsBenchmarks: gallery resolution 512×512, 8 LCM steps, CFG 2.
Training: 10,000 iterations.
License: CreativeML Open RAIL-M.










Comments