# Repurposing Geometric Foundation Models for Multi-view Diffusion arXiv: https://arxiv.org/abs/2603.22275 · HTML: https://mlsanity.com/paper/2603.22275/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. - Builds on 49 of its 56 references that are in the map. - First posted on arXiv 23 Mar 2026. ## Placed in - Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/) ## Builds on (most cited first, 30 of 49) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023) - [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023) - [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022) - [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022) - [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023) - [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020) - [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017) - [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021) - [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023) - [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022) - [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025) - [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025) - [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023) - [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023) - [Depth Anything 3: Recovering the Visual Space from Any Views](https://mlsanity.com/paper/2511.10647/) (2025) - [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021) - [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020) - [DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision](https://mlsanity.com/paper/2312.16256/) (2023) - [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020) - [CAT3D: Create Anything in 3D with Multi-View Diffusion Models](https://mlsanity.com/paper/2405.10314/) (2024) - [Depth Pro: Sharp Monocular Metric Depth in Less Than a Second](https://mlsanity.com/paper/2410.02073/) (2024) - [ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis](https://mlsanity.com/paper/2409.02048/) (2024) - [Differentiable Augmentation for Data-Efficient GAN Training](https://mlsanity.com/paper/2006.10738/) (2020) - [π3: Permutation-Equivariant Visual Geometry Learning](https://mlsanity.com/paper/2507.13347/) (2025) - [MapAnything: Universal Feed-Forward Metric 3D Reconstruction](https://mlsanity.com/paper/2509.13414/) (2025) ## Built on by (most cited first, 15 of 15) - [VGGT-Omega](https://mlsanity.com/paper/2605.15195/) (2026) - [PixWorld: Unifying 3D Scene Generation and Reconstruction in Pixel Space](https://mlsanity.com/paper/2607.05373/) (2026) - [Latent Riemannian Flow Matching for Geometry-Grounded 3D Foundation Models](https://mlsanity.com/paper/2607.19120/) (2026) - [Geometry-Aware Representation Denoising for Robust Multi-view 3D Reconstruction](https://mlsanity.com/paper/2605.26230/) (2026) - [VolFill: Single-View Amodal 3D Scene Reconstruction with Volumetric Flow Matching](https://mlsanity.com/paper/2605.31466/) (2026) - [DeblurNVS: Geometric Latent Diffusion for Novel View Synthesis from Sparse Motion-Blurred Images](https://mlsanity.com/paper/2606.01315/) (2026) - [GenRec: Knowing Where to Reconstruct and Where to Generate](https://mlsanity.com/paper/2608.17832/) (2026) - [USR-Drive: Unified Driving Scene Representation via Joint Denoising of 3D Gaussians and Boxes](https://mlsanity.com/paper/2608.19036/) (2026) - [GeoLAM: Learning Geometry-Grounded Latent Actions from Unlabeled Human Videos](https://mlsanity.com/paper/2609.17099/) (2026) - [GAE: Learning a Geometry-Native Latent Space for 3D-Consistent World Generation](https://mlsanity.com/paper/2609.24981/) (2026) - [InfiNoVA: Infinite Novel View Augmentation for Viewpoint Invariant Robot Policies](https://mlsanity.com/paper/2609.27734/) (2026) - [M3GD: Multi-Modal Multi-View Geometric Diffusion for Camera--LiDAR Novel View Synthesis](https://mlsanity.com/paper/2609.30056/) (2026) - [GeoVerse: World-Consistent Novel View Synthesis in Geometric Latent Space](https://mlsanity.com/paper/2609.35734/) (2026) - [SceneAE: Scene Generation and Reconstruction in Queryable 3D-Anchored Latent Space](https://mlsanity.com/paper/2610.19708/) (2026) - [Can Learning to Answer Spatial Questions Improve Novel-View Generation?](https://mlsanity.com/paper/2610.20610/) (2026) ## Abstract (arXiv) While recent advances in generative latent spaces have driven substantial progress in single-image generation, the optimal latent space for novel view synthesis (NVS) remains largely unexplored. In particular, NVS requires geometrically consistent generation across viewpoints, but existing approaches typically operate in a view-independent VAE latent space. In this paper, we propose Geometric Latent Diffusion (GLD), a framework that repurposes the geometrically consistent feature space of geometric foundation models as the latent space for multi-view diffusion. We show that these features not only support high-fidelity RGB reconstruction but also encode strong cross-view geometric correspondences, providing a well-suited latent space for NVS. Our experiments demonstrate that GLD outperforms both VAE and RAE on 2D image quality and 3D consistency metrics, while accelerating training by more than 4.4x compared to the VAE latent space. Notably, GLD remains competitive with state-of-the-art methods that leverage large-scale text-to-image pretraining, despite training its diffusion model from scratch without such generative pretraining.