ReTokenGS-stream-dl3dv
Streaming 3D Scene Reconstruction from Video Frames
2 min readupdated
The ReTokenGS-stream-dl3dv model solves the problem of fast 3D scene reconstruction from real-time incoming frames in a feed-forward mode (without additional per-scene optimization during testing). The architecture is based on a closed-loop system with external RMS norm (root mean square normalization) and latent gradient steps, fine-tuned on the DL3DV dataset. The output is a high-quality 3D representation of the scene with support for sequentially arriving views. This model will be useful for researchers and developers in computer vision and neural rendering.
Key features
- Streaming processing: views can arrive gradually in batches (for example, 8 + 4 + 4).
- No test-time optimization: fast scene generation without heavy fine-tuning.
- Trained on the DL3DV dataset with a fixed 50-frame window.
What you can do
- Reconstruct 3D scenes from sequentially incoming camera angles.
- Integrate the model into streaming neural rendering and video processing pipelines at a resolution of 256×256.
Example
The model accepts positioned incoming images and processes them through an iterative hidden state update loop. Detailed launch code and comparisons with baseline versions are available in the project repository on the stream_dl3dv branch.
Read more
Details• Architecture: loop with latent gradient step and external RMS norm
• Metrics (DL3DV-140, 16 views): PSNR 29.70 / SSIM 0.895 / LPIPS 0.176
• License: Apache-2.0 (based on LocusGS / TokenGS)
• Weight hash: 8aaa60c109ae6e5b7b869e0671278896de141fdc4fdde208d8989d4b3cd8c574










Comments