DESPINA lunar SDXL: Lunar surface views generation
2 min readupdated
This release is a set of trained weights based on Stable Diffusion XL designed to turn digital elevation models (DEMs) into realistic photographs of the lunar horizon. The system uses two ControlNet modules for depth and soft-edge analysis, along with a special LoRA add-on to reproduce an authentic lunar look based on Apollo archive imagery. The resulting horizon and topography strictly match the original geographic data. The tool is geared toward space researchers, planetary scientists, and visualization developers.
Key features
- Precise terrain anchoring: Uses an inverted depth map and contours to control generation pixel by pixel.
- Lunar sky: A specially trained LoRA supports rendering a black sky with correct noise.
- Comprehensive approach: Includes three components—a lunar LoRA for UNet, depth ControlNet, and soft-edge ControlNet.
- Scientific base: Built on academic research (ISPRS Annals) using ASU Apollo scans.
What you can do
- Landscape reconstruction: Create photorealistic panoramas of the lunar surface based on real elevation data (DEM).
- Batch rendering: Use the project's repository scripts to generate lunar views for specified camera coordinates and sun angles.
- Diffusers integration: Run generation directly via the Stable Diffusion XL ControlNet pipeline in Python scripts.
Example
import torch
from diffusers import ControlNetModel, StableDiffusionXLControlNetPipeline
repo = "RonInDune/despina-lunar-sdxl"
cns = [ControlNetModel.from_pretrained(repo, subfolder="controlnet-depth"),
ControlNetModel.from_pretrained(repo, subfolder="controlnet-softedge")]
pipe = StableDiffusionXLControlNetPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0", controlnet=cns).to("cuda")
pipe.load_lora_weights(repo, subfolder="lora", weight_name="pytorch_lora_weights.safetensors")
pipe.fuse_lora(lora_scale=0.9)
image = pipe(
"A realistic image of the Moon's surface, from the perspective of a rover. Black sky.",
image=[depth_image, softedge_image],
controlnet_conditioning_scale=[0.8, 0.5],
num_inference_steps=30, guidance_scale=7.0,
).images[0]Read more
Details• Accuracy: average horizon error Horizon RMSE 0.94 px at 512 px.
• Training: 3000 steps on 1277 frames in fp32 using diffusers 0.31.0.
• Limitations: non-commercial use (research and education) due to base model and NASA/JSC/ASU data licenses.










Comments