mlsanityData to 13 Oct 2026

GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation

arXiv 2403.14621 ↗First posted 21 Mar 2024cs.CVcited by 299

Not on the map. Why?

Cited by 58 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce GRM, a large-scale reconstructor capable of recovering a 3D asset from sparse-view images in around 0.1s. GRM is a feed-forward transformer-based model that efficiently incorporates multi-view information to translate the input pixels into pixel-aligned Gaussians, which are unprojected to create a set of densely distributed 3D Gaussians representing a scene. Together, our transformer architecture and the use of 3D Gaussians unlock a scalable and efficient reconstruction framework. Extensive experimental results demonstrate the superiority of our method over alternatives regarding both reconstruction quality and efficiency. We also showcase the potential of GRM in generative tasks, i.e., text-to-3D and image-to-3D, by integrating it with existing multi-view diffusion models. Our project website is at: https://justimyhxu.github.io/projects/grm/.

Key facts. Cited by 299 papers in the map (data through 13 Oct 2026). Builds on 103 of its 111 references that are in the map. First posted on arXiv 21 Mar 2024.

Builds on

103 of 111 references.

References · 103

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,952
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,111
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,082
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,028
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,213
  9. 2023Segment Anythingcited by 8,195
  10. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,170
  11. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,000
  12. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,712
38 more
  1. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,585
  2. 2020Longformer: The Long-Document Transformercited by 3,393
  3. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,019
  4. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,975
  5. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,687
  6. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,686
  7. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,225
  8. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,039
  9. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,581
  10. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,529
  11. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,486
  12. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,288
  13. 2022TensoRF: Tensorial Radiance Fieldscited by 1,277
  14. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,212
  15. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  16. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,202
  17. 2021Alias-Free Generative Adversarial Networkscited by 1,183
  18. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,150
  19. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 931
  20. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  21. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 843
  22. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 839
  23. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  24. 2023Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstructioncited by 756
  25. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 738
  26. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 735
  27. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 732
  28. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  29. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 700
  30. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 697
  31. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  32. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 686
  33. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 685
  34. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 673
  35. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 640
  36. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 605
  37. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  38. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 557

50 most cited of 103 All 103 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 102; 2025: 127; 2026: 70.

299 papers.

Sort:

By area:

  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,497
  2. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 424
  3. 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 277
  4. 2025AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Viewscited by 238
  5. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
  6. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  7. 2025FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Viewscited by 176
  8. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  9. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 152
  10. 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 144
  11. 2024MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Secondscited by 137
  12. 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Imagecited by 132
38 more
  1. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 115
  2. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 114
  3. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Viewscited by 112
  4. 2025Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attentioncited by 104
  5. 2024MeshXL: Neural Coordinate Field for Generative 3D Foundation Modelscited by 102
  6. 20243DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusioncited by 100
  7. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
  8. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 98
  9. 2024Meshtron: High-Fidelity, Artist-Like 3D Mesh Generation at Scalecited by 88
  10. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 86
  11. 2024EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generationcited by 85
  12. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 77
  13. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 77
  14. 2025SparseFlex: High-Resolution and Arbitrary-Topology 3D Shape Modelingcited by 77
  15. 2024Scaling Mesh Generation via Compressive Tokenizationcited by 72
  16. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  17. 2024SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglementcited by 65
  18. 2024You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scalecited by 65
  19. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 64
  20. 2024PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Modelscited by 60
  21. 2025Bolt3D: Generating 3D Scenes in Secondscited by 57
  22. 2024LaRa: Efficient Large-Baseline Radiance Fieldscited by 55
  23. 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusioncited by 55
  24. 2024MeshFormer: High-Quality Mesh Generation with 3D-Guided Reconstruction Modelcited by 55
  25. 2024InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Modelscited by 55
  26. 2024HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priorscited by 54
  27. 2024STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenescited by 53
  28. 2025Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatarscited by 53
  29. 2024GGHead: Fast and Generalizable 3D Gaussian Headscited by 51
  30. 2024DiffGS: Functional Gaussian Splatting Diffusioncited by 51
  31. 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Textcited by 50
  32. 2024PivotMesh: Generic 3D Mesh Generation via Pivot Vertices Guidancecited by 49
  33. 2025DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generationcited by 49
  34. 2024TEXGen: a Generative Diffusion Model for Mesh Texturescited by 46
  35. 20254DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videoscited by 46
  36. 2024Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Modelscited by 44
  37. 2024Generative Gaussian Splatting for Unbounded 3D City Generationcited by 43
  38. 2024FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D Reconstructioncited by 40

50 most cited of 299 All 299 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024LM-Gaussian: Boost Sparse-view 3D Gaussian Splatting with Large Model Priors
  2. 2024GaussianObject: High-Quality 3D Object Reconstruction from Four Views with Gaussian Splatting
  3. 2024Gaussian Scenes: Pose-Free Sparse-View Scene Reconstruction using Depth-Enhanced Diffusion Priors
  4. 2025Diffusion-Guided Gaussian Splatting for Large-Scale Unconstrained 3D Reconstruction and Novel View Synthesis
  5. 2025WaveletGaussian: Wavelet-domain Diffusion for Sparse-view 3D Gaussian Object Reconstruction
  6. 2024GSD: View-Guided Gaussian Splatting Diffusion for 3D Reconstruction
  7. 2025Generative Gaussian Splatting: Generating 3D Scenes with Video Diffusion Priors
  8. 2024GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splatting