# GenFusion: Closing the Loop between Reconstruction and Generation via Videos

arXiv: https://arxiv.org/abs/2503.21219 · HTML: https://mlsanity.com/paper/2503.21219/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 67 references that are in the map.
- First posted on arXiv 27 Mar 2025.

## Builds on (most cited first, 30 of 59)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [2D Gaussian Splatting for Geometrically Accurate Radiance Fields](https://mlsanity.com/paper/2403.17888/) (2024)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [Depth-supervised NeRF: Fewer Views and Faster Training for Free](https://mlsanity.com/paper/2107.02791/) (2021)
- [pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction](https://mlsanity.com/paper/2312.12337/) (2023)
- [Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields](https://mlsanity.com/paper/2304.06706/) (2023)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs](https://mlsanity.com/paper/2112.00724/) (2021)
- [MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2403.14627/) (2024)
- [DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision](https://mlsanity.com/paper/2312.16256/) (2023)
- [DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors](https://mlsanity.com/paper/2310.12190/) (2023)
- [MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction](https://mlsanity.com/paper/2206.00665/) (2022)
- [Baking Neural Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2103.14645/) (2021)
- [Neuralangelo: High-Fidelity Neural Surface Reconstruction](https://mlsanity.com/paper/2306.03092/) (2023)
- [CAT3D: Create Anything in 3D with Multi-View Diffusion Models](https://mlsanity.com/paper/2405.10314/) (2024)
- [ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis](https://mlsanity.com/paper/2409.02048/) (2024)
- [ReconFusion: 3D Reconstruction with Diffusion Priors](https://mlsanity.com/paper/2312.02981/) (2023)
- [FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization](https://mlsanity.com/paper/2303.07418/) (2023)
- [Dense Depth Priors for Neural Radiance Fields from Sparse Input Views](https://mlsanity.com/paper/2112.03288/) (2021)
- [Novel View Synthesis with Diffusion Models](https://mlsanity.com/paper/2210.04628/) (2022)
- [DepthSplat: Connecting Gaussian Splatting and Depth](https://mlsanity.com/paper/2410.13862/) (2024)
- [FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting](https://mlsanity.com/paper/2312.00451/) (2023)

## Built on by (most cited first, 30 of 41)

- [WorldMem: Long-term Consistent World Simulation with Memory](https://mlsanity.com/paper/2504.12369/) (2025)
- [GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priors](https://mlsanity.com/paper/2508.09667/) (2025)
- [Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Survey](https://mlsanity.com/paper/2507.14501/) (2025)
- [GSFix3D: Diffusion-Guided Repair of Novel Views in Gaussian Splatting](https://mlsanity.com/paper/2508.14717/) (2025)
- [World-R1: Reinforcing 3D Constraints for Text-to-Video Generation](https://mlsanity.com/paper/2604.24764/) (2026)
- [G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior](https://mlsanity.com/paper/2510.12099/) (2025)
- [GaussFusion: Improving 3D Reconstruction in the Wild with A Geometry-Informed Video Generator](https://mlsanity.com/paper/2603.25053/) (2026)
- [A Mechanistic View on Video Generation as World Models: State and Dynamics](https://mlsanity.com/paper/2601.17067/) (2026)
- [ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models](https://mlsanity.com/paper/2604.12251/) (2026)
- [ProSplat: Improved Feed-Forward 3D Gaussian Splatting for Wide-Baseline Sparse Views](https://mlsanity.com/paper/2506.07670/) (2025)
- [Sparse-View 3D Reconstruction: Recent Advances and Open Challenges](https://mlsanity.com/paper/2507.16406/) (2025)
- [GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstruction](https://mlsanity.com/paper/2512.25073/) (2025)
- [GIFSplat: Generative Prior-Guided Iterative Feed-Forward 3D Gaussian Splatting from Sparse Views](https://mlsanity.com/paper/2602.22571/) (2026)
- [Pantheon360: Taming Digital Twin Generation via 3D-Aware 360deg Video Diffusion](https://mlsanity.com/paper/2605.25449/) (2026)
- [FixingGS: Enhancing 3D Gaussian Splatting via Training-Free Score Distillation](https://mlsanity.com/paper/2509.18759/) (2025)
- [WaveletGaussian: Wavelet-domain Diffusion for Sparse-view 3D Gaussian Object Reconstruction](https://mlsanity.com/paper/2509.19073/) (2025)
- [Self-Evolving 3D Scene Generation from a Single Image](https://mlsanity.com/paper/2512.08905/) (2025)
- [Breaking the Vicious Cycle: Coherent 3D Gaussian Splatting from Sparse and Motion-Blurred Views](https://mlsanity.com/paper/2512.10369/) (2025)
- [GeoNVS: Geometry Grounded Video Diffusion for Novel View Synthesis](https://mlsanity.com/paper/2603.14965/) (2026)
- [Droplet3D: Commonsense Priors from Videos Facilitate 3D Generation](https://mlsanity.com/paper/2508.20470/) (2025)
- [ConfCtrl: Enabling Precise Camera Control in Video Diffusion via Confidence-Aware Interpolation](https://mlsanity.com/paper/2603.09819/) (2026)
- [GSCompleter: A Distillation-Free Plugin for Metric-Aware 3D Gaussian Splatting Completion in Seconds](https://mlsanity.com/paper/2604.20155/) (2026)
- [HAD: Hallucination-Aware Diffusion Priors for 3D Reconstruction](https://mlsanity.com/paper/2605.16873/) (2026)
- [RealityBridge: Bridging Editable 3D Gaussian Splatting Driving Simulations and Real-World Videos](https://mlsanity.com/paper/2606.16278/) (2026)
- [Latent Riemannian Flow Matching for Geometry-Grounded 3D Foundation Models](https://mlsanity.com/paper/2607.19120/) (2026)
- [R2S-EGO: Dual-Proxy Refinement for Sparse-Capture Real-to-Sim](https://mlsanity.com/paper/2608.06827/) (2026)
- [SCORP: Scene-Consistent Object Refinement via Proxy Generation and Tuning](https://mlsanity.com/paper/2506.23835/) (2025)
- [Pi-GS: Sparse-View Gaussian Splatting with Dense π3 Initialization](https://mlsanity.com/paper/2602.03327/) (2026)
- [CrowdGaussian: Reconstructing High-Fidelity 3D Gaussians for Human Crowd from a Single Image](https://mlsanity.com/paper/2603.17779/) (2026)
- [Can These Views Be One Scene? Evaluating Multiview 3D Consistency when 3D Foundation Models Hallucinate](https://mlsanity.com/paper/2605.18754/) (2026)

## Abstract (arXiv)

Recently, 3D reconstruction and generation have demonstrated impressive novel view synthesis results, achieving high fidelity and efficiency. However, a notable conditioning gap can be observed between these two fields, e.g., scalable 3D scene reconstruction often requires densely captured views, whereas 3D generation typically relies on a single or no input view, which significantly limits their applications. We found that the source of this phenomenon lies in the misalignment between 3D constraints and generative priors. To address this problem, we propose a reconstruction-driven video diffusion model that learns to condition video frames on artifact-prone RGB-D renderings. Moreover, we propose a cyclical fusion pipeline that iteratively adds restoration frames from the generative model to the training set, enabling progressive expansion and addressing the viewpoint saturation limitations seen in previous reconstruction and generation pipelines. Our evaluation, including view synthesis from sparse view and masked input, validates the effectiveness of our approach. More details at https://genfusion.sibowu.com.