Key takeaways
- L3G-World pairs a latent 3D Gaussian memory with a video diffusion transformer, and the paper reports 9.02 FPS at 480x832 on one H100.
- On DL3DV the memory holds 33K Gaussians in 11.88 MiB, which the paper says is 266 to 417 times fewer primitives than NeoVerse.
- The method handles static scenes only, and the GitHub repository had no code or weights released as of October 31, 2026.
L3G-World is a research system from KAIST that generates an explorable world as video in real time, and keeps the scenery consistent when the camera comes back to a place it has already shown. The paper reports 9.02 frames per second at 480x832 on a single H100 GPU. Its trick is a compact 3D memory that stores what the model has already generated, in the same compressed form the video model works in.
Why this matters
World models are video generators you can steer. You give a starting image and a camera path, and the model produces what you would see. The hard part is memory. Walk around a corner and come back, and a weak model paints a different room.
Existing fixes cost a lot. The authors say methods that feed past frames back in as context grow in memory and attention cost with the length of the stream. Methods that keep an explicit 3D memory usually render it to a picture and then re-encode the picture with a VAE (the network that compresses frames into the latents a video model reads), and that step blocks real-time use. L3G-World tries to remove both costs.
What the team built
The system has two parts that take turns, one chunk of video at a time.
L3G-Mem is the memory. It is a set of 3D Gaussians, small blobs of color and position that can be rendered quickly from any camera angle. Each Gaussian stores a VAE latent feature instead of a color. A feed-forward module adds new Gaussians only where the scene is not yet covered or the error is high, and nudges existing ones with residual updates. Because of this, memory size follows how much space has been explored, not how long the video runs.
L3G-Gen is the generator. It is a video diffusion transformer built on Cosmos-Predict2.5 and fine-tuned with LoRA. It reads latents rendered straight from the memory for the next camera positions, so nothing has to be re-encoded. The team trained it with rectified flow, then distilled it into a few-step student with AnyFlow to reach real-time speed.
Training used DL3DV for both parts and RealEstate10K for the generator.
What the results show
The authors tested 481-frame trajectories that loop back to the start, on DL3DV and RealEstate10K. They compared against temporal-context models (Yume 1.5, Matrix-Game 3.0), point-cloud memory methods (GEN3C, Spatia, SPMem, Lyra 2.0, 3DScenePrompt) and a 3D Gaussian method (NeoVerse).
The efficiency numbers are the headline. On DL3DV the full model's memory holds 33K Gaussians and takes 11.88 MiB. Compared with NeoVerse, the paper reports 266 to 417 times fewer primitives and 42 to 67 times less memory. Against Lyra 2.0 it reports 8.9 to 9.5 times faster memory reconstruction and 7.8 to 10.4 times less memory.
Quality numbers are mixed and worth reading closely. On DL3DV the model scores a PSNR of 16.00, an FID of 40.57 and a camera translation error of 0.142. On RealEstate10K it scores a PSNR of 18.36 and an FID of 36.09. The paper also offers a "Fast" variant with more Gaussians that scores lower on most of these: on DL3DV, PSNR 14.59 and FID 43.05.
Ablations support the design. Without adaptive spawning, the memory on DL3DV grows from 41K to 75K Gaussians. Without the step that lifts VAE features into the Gaussians, camera error and FID get worse in the full-system test.
One caution: the paper's abstract and introduction give slightly different ranges for the primitive-count savings, so treat the exact multiples as the authors' claims and check the tables.
What the authors say it can't do
The limitations section lists three.
- It handles static scenes only. The memory does not model objects that move on their own.
- Memory still grows with the area explored, and can get large in very big spaces. The authors suggest pruning or a hierarchical memory.
- Errors made during long generation can still carry into later memory updates, even though refinement corrects some of them.
All results are the authors' own, on two datasets, and the paper has not been through the checks other labs would apply by reproducing it.
Code and availability
The GitHub repository lists four planned releases: L3G-Mem code and checkpoints, L3G-Gen code and checkpoints, inference code, and the Fast checkpoints. When we checked on October 31, none were marked done and the repository held only the README, licenses and an overview figure. The README says the L3G-Mem weights will be under CC BY-NC 4.0, which bars commercial use, and the generator weights under the NVIDIA Open Model License. More videos are on the project page.
Watch for the checkpoints. Until they land, nobody outside the team can test whether the speed and consistency hold up.
Frequently asked questions
What is L3G-World?
L3G-World is a world generation system from KAIST that creates video of an explorable scene from an image, text and camera path. It keeps a latent 3D Gaussian memory so places look the same when revisited.
Is the L3G-World code available?
Not yet, as of October 31, 2026. The GitHub README lists code and checkpoints as planned releases, and the repository held only the README, licenses and an overview figure.
What is a latent 3D Gaussian memory?
It is a set of 3D Gaussians that each store a VAE latent feature instead of a color. The video model can read latents rendered from it directly, with no re-encoding step.
Can L3G-World handle moving objects?
No. The authors say the memory does not model independently moving objects and list separating dynamic content from static geometry as future work.