mlsanityData to 7 Oct 2026

L4GM: Large 4D Gaussian Reconstruction Model

arXiv 2406.10324 ↗First posted 14 Jun 2024cs.CV cs.LGcited by 143

Not on the map. Why?

Cited by 31 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present L4GM, the first 4D Large Reconstruction Model that produces animated objects from a single-view video input -- in a single feed-forward pass that takes only a second. Key to our success is a novel dataset of multiview videos containing curated, rendered animated objects from Objaverse. This dataset depicts 44K diverse objects with 110K animations rendered in 48 viewpoints, resulting in 12M videos with a total of 300M frames. We keep our L4GM simple for scalability and build directly on top of LGM, a pretrained 3D Large Reconstruction Model that outputs 3D Gaussian ellipsoids from multiview image input. L4GM outputs a per-frame 3D Gaussian Splatting representation from video frames sampled at a low fps and then upsamples the representation to a higher fps to achieve temporal smoothness. We add temporal self-attention layers to the base LGM to help it learn consistency across time, and utilize a per-timestep multiview rendering loss to train the model. The representation is upsampled to a higher framerate by training an interpolation model which produces intermediate 3D Gaussian representations. We showcase that L4GM that is only trained on synthetic data generalizes extremely well on in-the-wild videos, producing high quality animated 3D assets.

Key facts. Cited by 143 papers in the map (data through 7 Oct 2026). Builds on 61 of its 72 references that are in the map. First posted on arXiv 14 Jun 2024.

Builds on

61 of 72 references.

References · 61

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  10. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  11. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  12. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
38 more
  1. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  2. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  4. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  5. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  6. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  7. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  8. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  9. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  10. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  11. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  12. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 742
  13. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
  14. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  15. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  16. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  17. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
  18. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 553
  19. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  20. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  21. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 344
  22. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 330
  23. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  24. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 298
  25. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  26. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  27. 2023Text-To-4D Dynamic Scene Generationcited by 204
  28. 2023Segment and Track Anythingcited by 193
  29. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
  30. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  31. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  32. 2023Text-to-3D with Classifier Score Distillationcited by 127
  33. 2023Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Videocited by 124
  34. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
  35. 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 115
  36. 2022Is Attention All That NeRF Needs?cited by 113
  37. 2023Animate124: Animating One Image to 4D Dynamic Scenecited by 113
  38. 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 21; 2025: 67; 2026: 55.

143 papers.

Sort:

By area:

  1. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  2. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  3. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  4. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 143
  5. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 98
  6. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 68
  7. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 66
  8. 2024FlashSplat: 2D to 3D Gaussian Splatting Segmentation Solved Optimallycited by 57
  9. 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusioncited by 55
  10. 2025Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatarscited by 52
  11. 2024GenXD: Generating Any 3D and 4D Scenescited by 49
  12. 2024Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videoscited by 48
38 more
  1. 20254DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videoscited by 46
  2. 2025SV4D 2.0: Enhancing Spatio-Temporal Consistency in Multi-View Video Diffusion for High-Quality 4D Generationcited by 44
  3. 2025Puppeteer: Rig and Animate Your 3D Modelscited by 41
  4. 2025Motion Anything: Any to Motion Generationcited by 40
  5. 2024DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenescited by 37
  6. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 37
  7. 2024DreamDrive: Generative 4D Scene Modeling from Street View Imagescited by 35
  8. 2025Free4D: Tuning-free 4D Scene Generation with Spatial-Temporal Consistencycited by 34
  9. 2025Gaussian Variation Field Diffusion for High-fidelity Video-to-4D Synthesiscited by 33
  10. 2025Any4D: Unified Feed-Forward Metric 4D Reconstructioncited by 32
  11. 2025AnimateAnyMesh: A Feed-Forward 4D Foundation Model for Text-Driven Universal Mesh Animationcited by 31
  12. 20254DNeX: Feed-Forward 4D Generative Modeling Made Easycited by 31
  13. 2024DrivingRecon: Large 4D Gaussian Reconstruction Model For Autonomous Drivingcited by 27
  14. 2025DreamArt: Generating Interactable Articulated Objects from a Single Imagecited by 25
  15. 2024Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attentioncited by 24
  16. 2024Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perceptioncited by 23
  17. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 23
  18. 2025UniLat3D: Geometry-Appearance Unified Latents for Single-Stage 3D Generationcited by 20
  19. 2025ShapeGen4D: Towards High Quality 4D Shape Generation from Videoscited by 20
  20. 2024Generative Densification: Learning to Densify Gaussians for High-Fidelity Generalizable 3D Reconstructioncited by 18
  21. 2025Vid2Sim: Generalizable, Video-based Reconstruction of Appearance, Geometry and Physics for Mesh-free Simulationcited by 18
  22. 2025Compositional Generative Model of Unbounded 4D Citiescited by 16
  23. 2025PartRM: Modeling Part-Level Dynamics with Large Cross-State Reconstruction Modelcited by 15
  24. 2025AnimaX: Animating the Inanimate in 3D with Joint Video-Pose Diffusion Modelscited by 15
  25. 2026Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesiscited by 15
  26. 2026ActionMesh: Animated 3D Mesh Generation with Temporal 3D Diffusioncited by 15
  27. 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14
  28. 2025DGS-LRM: Real-Time Deformable 3D Gaussian Reconstruction From Monocular Videoscited by 14
  29. 2025Feature4X: Bridging Any Monocular Video to 4D Agentic AI with Versatile Gaussian Feature Fieldscited by 13
  30. 2025Articulated Kinematics Distillation from Video Diffusion Modelscited by 13
  31. 20254D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Timecited by 13
  32. 2025Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videoscited by 13
  33. 20254Real-Video-V2: Fused View-Time Attention and Feedforward Reconstruction for 4D Scene Generationcited by 12
  34. 2026Mesh4D: 4D Mesh Reconstruction and Tracking from Monocular Videocited by 12
  35. 2025Vivid4D: Improving 4D Reconstruction from Monocular Video by Video Inpaintingcited by 11
  36. 2026TokenGS: Decoupling 3D Gaussian Prediction from Pixels with Learnable Tokenscited by 11
  37. 2024DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstructioncited by 10
  38. 2025Not All Frame Features Are Equal: Video-to-4D Generation via Decoupling Dynamic-Static Featurescited by 10

50 most cited of 143 All 143 (JSON)