mlsanityData to 7 Oct 2026

ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Model

arXiv 2408.16767 ↗First posted 29 Aug 2024cs.CV cs.AI cs.GRcited by 139

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Advancements in 3D scene reconstruction have transformed 2D images from the real world into 3D models, producing realistic 3D results from hundreds of input photos. Despite great success in dense-view reconstruction scenarios, rendering a detailed scene from insufficient captured views is still an ill-posed optimization problem, often resulting in artifacts and distortions in unseen areas. In this paper, we propose ReconX, a novel 3D scene reconstruction paradigm that reframes the ambiguous reconstruction challenge as a temporal generation task. The key insight is to unleash the strong generative prior of large pre-trained video diffusion models for sparse-view reconstruction. However, 3D view consistency struggles to be accurately preserved in directly generated video frames from pre-trained models. To address this, given limited input views, the proposed ReconX first constructs a global point cloud and encodes it into a contextual space as the 3D structure condition. Guided by the condition, the video diffusion model then synthesizes video frames that are both detail-preserved and exhibit a high degree of 3D consistency, ensuring the coherence of the scene from various perspectives. Finally, we recover the 3D scene from the generated video through a confidence-aware 3D Gaussian Splatting optimization scheme. Extensive experiments on various real-world datasets show the superiority of our ReconX over state-of-the-art methods in terms of quality and generalizability.

Key facts. Cited by 139 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 57 of its 88 references that are in the map. First posted on arXiv 29 Aug 2024.

Builds on

57 of 88 references.

Closest ancestors · 13

Same direction, newest first

  1. 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image30 May
  2. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Models16 May
  3. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion18 Mar
  4. 2024V3D: Video Diffusion Models are Effective 3D Generators11 Mar
  5. 2023Sherpa3D: Boosting High-Fidelity Text-to-3D Generation via Coarse 3D Prior11 Dec
  6. 2023ReconFusion: 3D Reconstruction with Diffusion Priors5 Dec
  7. 2023ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image27 Oct
  8. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  9. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  10. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  11. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  12. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
1 more
  1. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov

Other references · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  12. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
32 more
  1. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  2. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  4. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  5. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  6. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 667
  7. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  8. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 553
  9. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
  10. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  11. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  12. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  13. 2023FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularizationcited by 310
  14. 2023FSGS: Real-Time Few-shot View Synthesis using Gaussian Splattingcited by 278
  15. 2024DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalizationcited by 256
  16. 2023SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesiscited by 247
  17. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  18. 2021Vision-Only Robot Navigation in a Neural Radiance Worldcited by 153
  19. 2023SparseGS: Sparse View Synthesis using 3D Gaussian Splattingcited by 134
  20. 2022Generalizable Patch-Based Neural Renderingcited by 121
  21. 2024latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstructioncited by 114
  22. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 113
  23. 2023Learning to Render Novel Views from Wide-Baseline Stereo Pairscited by 103
  24. 2024NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wildcited by 85
  25. 2024Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusioncited by 76
  26. 2023MuRF: Multi-Baseline Radiance Fieldscited by 70
  27. 2024InstantSplat: Sparse-view Gaussian Splatting in Secondscited by 67
  28. 2024Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Reviewcited by 40
  29. 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 39
  30. 2024Make-Your-3D: Fast and Consistent Subject-Driven 3D Content Generationcited by 32
  31. 20173D Face Reconstruction with Geometry Details from a Single Imagecited by 27
  32. 2022Single-view 3D Mesh Reconstruction for Seen and Unseen Categoriescited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 21; 2025: 75; 2026: 43.

139 papers.

Sort:

By area:

  1. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  2. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
  3. 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 152
  4. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 152
  5. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 139
  6. 2024MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Secondscited by 136
  7. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Viewscited by 111
  8. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 98
  9. 2024You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scalecited by 65
  10. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 62
  11. 2025Bolt3D: Generating 3D Scenes in Secondscited by 57
  12. 2024InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Modelscited by 53
38 more
  1. 2024StreetCrafter: Street View Synthesis with Controllable Video Diffusion Modelscited by 49
  2. 2025PhysAnimator: Physics-Guided Generative Cartoon Animationcited by 40
  3. 2025LangSplatV2: High-dimensional 3D Language Gaussian Splatting with 450+ FPScited by 40
  4. 2025Video-T1: Test-Time Scaling for Video Generationcited by 39
  5. 2025FlexWorld: Progressively Expanding 3D Scenes for Flexiable-View Synthesiscited by 37
  6. 2025VRAG: Learning World Models for Interactive Video Generationcited by 37
  7. 2025GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priorscited by 36
  8. 2025ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understandingcited by 34
  9. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generationcited by 33
  10. 2025Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Controlcited by 33
  11. 2025Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrationscited by 33
  12. 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Predictioncited by 33
  13. 2025EX-4D: EXtreme Viewpoint 4D Video Synthesis via Depth Watertight Meshcited by 32
  14. 2024MVGenMaster: Scaling Multi-View Generation from Any Image via 3D Priors Enhanced Diffusion Modelcited by 31
  15. 2024FreeSim: Toward Free-viewpoint Camera Simulation in Driving Scenescited by 30
  16. 2025Decompositional Neural Scene Reconstruction with Generative Diffusion Priorcited by 30
  17. 2025Emergent Temporal Correspondences from Video Diffusion Transformerscited by 30
  18. 2025NANO3D: A Training-Free Approach for Efficient 3D Editing Without Maskscited by 29
  19. 2024DepthLab: From Partial to Completecited by 27
  20. 2025VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in One Stepcited by 27
  21. 2025StarGen: A Spatiotemporal Autoregression Framework with Video Diffusion Model for Scalable and Controllable Scene Generationcited by 26
  22. 2025WorldExplorer: Towards Generating Fully Navigable 3D Scenescited by 26
  23. 2024Splatter-360: Generalizable 360circ Gaussian Splatting for Wide-baseline Panoramic Imagescited by 23
  24. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 23
  25. 2025Zero-Shot Monocular Scene Flow Estimation in the Wildcited by 20
  26. 2025EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidancecited by 20
  27. 2025WorldGen: From Text to Traversable and Interactive 3D Worldscited by 18
  28. 2024SLGaussian: Fast Language Gaussian Splatting in Sparse Viewscited by 17
  29. 2025SpatialCrafter: Unleashing the Imagination of Video Diffusion Models for Scene Reconstruction from Limited Observationscited by 16
  30. 2024Using Diffusion Priors for Video Amodal Segmentationcited by 15
  31. 2025EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memorycited by 14
  32. 2025GeoVideo: Introducing Geometric Regularization into Video Generation Modelcited by 14
  33. 2025SteerX: Creating Any Camera-Free 3D and 4D Scenes with Geometric Steeringcited by 13
  34. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 13
  35. 2024NVComposer: Boosting Generative Novel View Synthesis with Multiple Sparse and Unposed Imagescited by 12
  36. 2024Latent-Reframe: Enabling Camera Control for Video Diffusion Model without Trainingcited by 12
  37. 2025VideoRFSplat: Direct Scene-Level Text-to-3D Gaussian Splatting Generation with Flexible Pose and Multi-View Joint Modelingcited by 12
  38. 2025Quantifying and Alleviating Co-Adaptation in Sparse-View 3D Gaussian Splattingcited by 12

50 most cited of 139 All 139 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026ReconSplat: Generalizable 3D Scene Reconstruction Beyond Observed Views
  2. 2026AnyRecon: Arbitrary-View 3D Reconstruction with Video Diffusion Model
  3. 2024VI3DRM:Towards meticulous 3D Reconstruction from Sparse Views via Photo-Realistic Novel View Synthesis
  4. 2024MVDiff: Scalable and Flexible Multi-View Diffusion for 3D Object Reconstruction from Single-View
  5. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis
  6. 2026Training-Free Instance-Aware 3D Scene Reconstruction and Diffusion-Based View Synthesis from Sparse Images
  7. 2023Sparse3D: Distilling Multiview-Consistent Diffusion for Object Reconstruction from Sparse Views
  8. 2026Gen3R: 3D Scene Generation Meets Feed-Forward Reconstruction