mlsanityData to 14 Oct 2026

Wonderland: Navigating 3D Scenes from a Single Image

arXiv 2412.12091 ↗First posted 16 Dec 2024cs.CVcited by 99

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

How can one efficiently generate high-quality, wide-scope 3D scenes from arbitrary single images? Existing methods suffer several drawbacks, such as requiring multi-view data, time-consuming per-scene optimization, distorted geometry in occluded areas, and low visual quality in backgrounds. Our novel 3D scene reconstruction pipeline overcomes these limitations to tackle the aforesaid challenge. Specifically, we introduce a large-scale reconstruction model that leverages latents from a video diffusion model to predict 3D Gaussian Splattings of scenes in a feed-forward manner. The video diffusion model is designed to create videos precisely following specified camera trajectories, allowing it to generate compressed video latents that encode multi-view information while maintaining 3D consistency. We train the 3D reconstruction model to operate on the video latent space with a progressive learning strategy, enabling the efficient generation of high-quality, wide-scope, and generic 3D scenes. Extensive evaluations across various datasets affirm that our model significantly outperforms existing single-view 3D scene generation methods, especially with out-of-domain images. Thus, we demonstrate for the first time that a 3D reconstruction model can effectively be built upon the latent space of a diffusion model in order to realize efficient 3D scene generation.

Key facts. Cited by 99 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 104 of its 124 references that are in the map. First posted on arXiv 16 Dec 2024.

Builds on

104 of 124 references.

Closest ancestors · 28

Same direction, newest first

  1. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion7 Nov
  2. 2024VistaDream: Sampling multiview consistent images for single-view scene reconstruction22 Oct
  3. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis3 Sep
  4. 2024ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Model29 Aug
  5. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency24 Jul
  6. 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusion16 Jul
  7. 2024STAR: Skeleton-aware Text-based 4D Avatar Generation with In-Network Motion Retargeting7 Jun
  8. 2024Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Models26 May
  9. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Models16 May
  10. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusion10 Apr
  11. 2024Comp4D: LLM-Guided Compositional 4D Scene Generation25 Mar
  12. 2024VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models18 Mar
16 more
  1. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion18 Mar
  2. 2024GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splatting15 Mar
  3. 2024V3D: Video Diffusion Models are Effective 3D Generators11 Mar
  4. 2023ReconFusion: 3D Reconstruction with Diffusion Priors5 Dec
  5. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion14 Nov
  6. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model10 Nov
  7. 2023ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image27 Oct
  8. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  9. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  10. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
  11. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  12. 2023NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models19 Apr
  13. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
  14. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  15. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  16. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 76

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  10. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  11. 2022Scalable Diffusion Models with Transformerscited by 6,054
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
38 more
  1. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  3. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  4. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  5. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  6. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  7. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  8. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  9. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  10. 2017Mixed Precision Trainingcited by 1,156
  11. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  12. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  13. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  14. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  15. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 684
  16. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  17. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  18. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 566
  19. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
  20. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 535
  21. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  22. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  23. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
  24. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 364
  25. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 349
  26. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 301
  27. 2023SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesiscited by 247
  28. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 243
  29. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 232
  30. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 231
  31. 2016Generating Images with Perceptual Similarity Metrics based on Deep Networkscited by 222
  32. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 202
  33. 2024LVSM: A Large View Synthesis Model with Minimal 3D Inductive Biascited by 189
  34. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 185
  35. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 180
  36. 2024CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generationcited by 166
  37. 2023WonderJourney: Going from Anywhere to Everywherecited by 158
  38. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 52; 2026: 47.

99 papers.

Sort:

By area:

  1. 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 168
  2. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 125
  3. 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Modelscited by 98
  4. 2025Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generationcited by 77
  5. 2025Bolt3D: Generating 3D Scenes in Secondscited by 59
  6. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 44
  7. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 41
  8. 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Predictioncited by 38
  9. 2025FlexWorld: Progressively Expanding 3D Scenes for Flexiable-View Synthesiscited by 37
  10. 2025Matrix-3D: Omnidirectional Explorable 3D World Generationcited by 35
  11. 2025FlashWorld: High-quality 3D Scene Generation within Secondscited by 32
  12. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 27
38 more
  1. 2025VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generationcited by 25
  2. 20253D Scene Generation: A Surveycited by 24
  3. 2026VGGRPO: Towards World-Consistent Video Generation with 4D Latent Rewardcited by 24
  4. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modelingcited by 21
  5. 2025PlayerOne: Egocentric World Simulatorcited by 20
  6. 2025Positional Encoding Fieldcited by 20
  7. 2025Sharp Monocular View Synthesis in Less Than a Secondcited by 20
  8. 2025Generative Gaussian Splatting: Generating 3D Scenes with Video Diffusion Priorscited by 19
  9. 2025A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generationcited by 19
  10. 2025WorldGen: From Text to Traversable and Interactive 3D Worldscited by 19
  11. 2025Taming Video Models for 3D and 4D Generation via Zero-Shot Camera Controlcited by 18
  12. 2026WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memoriescited by 18
  13. 2025SpatialCrafter: Unleashing the Imagination of Video Diffusion Models for Scene Reconstruction from Limited Observationscited by 16
  14. 2025WorldGrow: Generating Infinite 3D Worldcited by 16
  15. 2025Precise Action-to-Video Generation Through Visual Action Promptscited by 15
  16. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 15
  17. 2025VideoRFSplat: Direct Scene-Level Text-to-3D Gaussian Splatting Generation with Flexible Pose and Multi-View Joint Modelingcited by 14
  18. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applicationscited by 14
  19. 2025A Recipe for Generating 3D Worlds From a Single Imagecited by 13
  20. 2025Cross-Frame Representation Alignment for Fine-Tuning Video Diffusion Modelscited by 12
  21. 2025Aerial World Model for Long-horizon Visual Generation and Navigation in 3D Spacecited by 12
  22. 2026Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied Simulationcited by 12
  23. 2025Text-to-3D by Stitching a Multi-view Reconstruction Network to a Video Generatorcited by 11
  24. 2025How Much 3D Do Video Foundation Models Encode?cited by 11
  25. 2026One2Scene: Geometric Consistent Explorable 3D Scene Generation from a Single Imagecited by 11
  26. 2026Prisma-World: Camera-Controllable Multi-Agent Video World Modelcited by 11
  27. 2025Can Video Diffusion Model Reconstruct 4D Geometry?cited by 10
  28. 2026SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generationcited by 10
  29. 2026LagerNVS: Latent Geometry for Fully Neural Real-time Novel View Synthesiscited by 9
  30. 2025Scaling Sequence-to-Sequence Generative Neural Renderingcited by 8
  31. 2026World Reconstruction From Inconsistent Viewscited by 8
  32. 2026WorldClaw: Agentic 3D Open-World Generation at Scalecited by 8
  33. 2025IDCNet: Guided Video Diffusion for Metric-Consistent RGBD Scene Generation with Precise Camera Controlcited by 7
  34. 2025PanoWorld-X: Generating Explorable Panoramic Worlds via Sphere-Aware Video Diffusioncited by 7
  35. 2025Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Modelscited by 7
  36. 2025Martian World Model: Controllable Video Synthesis with Physically Accurate 3D Reconstructionscited by 6
  37. 2025PostCam: Camera-Controllable Novel-View Video Generation with Query-Shared Cross-Attentioncited by 6
  38. 2025I-Scene: 3D Instance Models are Implicit Generalizable Spatial Learnerscited by 6

50 most cited of 99 All 99 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Sampling 3D Gaussian Scenes in Seconds with Latent Diffusion Models
  2. 2026ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models
  3. 2025G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior
  4. 2025GaussVideoDreamer: 3D Scene Generation with Video Diffusion and Inconsistency-Aware Gaussian Splatting
  5. 2024Baking Gaussian Splatting into Diffusion Denoiser for Fast and Scalable Single-stage Image-to-3D Generation and Reconstruction
  6. 2025VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in One Step
  7. 2026GaussVid: Sparse-View Gaussian Splatting with 3D-Aware Video Diffusion Priors
  8. 2025Scene Splatter: Momentum 3D Scene Generation from Single Image with Video Diffusion Model