Advances in 4D Generation: A Survey
Not on the map. Why?
Abstract (arXiv)
Generative artificial intelligence has recently progressed from static image and video synthesis to 3D content generation, culminating in the emergence of 4D generation-the task of synthesizing temporally coherent dynamic 3D assets guided by user input. As a burgeoning research frontier, 4D generation enables richer interactive and immersive experiences, with applications ranging from digital humans to autonomous driving. Despite rapid progress, the field lacks a unified understanding of 4D representations, generative frameworks, basic paradigms, and the core technical challenges it faces. This survey provides a systematic and in-depth review of the 4D generation landscape. To comprehensively characterize 4D generation, we first categorize fundamental 4D representations and outline associated techniques for 4D generation. We then present an in-depth analysis of representative generative pipelines based on conditions and representation methods. Subsequently, we discuss how motion and geometry priors are integrated into 4D outputs to ensure spatio-temporal consistency under various control schemes. From an application perspective, this paper summarizes 4D generation tasks in areas such as dynamic object/scene generation, digital human synthesis, editable 4D content, and embodied AI. Furthermore, we summarize and multi-dimensionally compare four basic paradigms for 4D generation: End-to-End, Generated-Data-Based, Implicit-Distillation-Based, and Explicit-Supervision-Based. Concluding our analysis, we highlight five key challenges-consistency, controllability, diversity, efficiency, and fidelity-and contextualize these with current approaches.By distilling recent advances and outlining open problems, this work offers a comprehensive and forward-looking perspective to guide future research in 4D generation.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 136 of its 176 references that are in the map. First posted on arXiv 18 Mar 2025.
Builds on
136 of 176 references.
References · 136
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
38 more
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2022Video Diffusion Modelscited by 2,280
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
- 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
- 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
- 2018World Modelscited by 1,444
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
- 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
- 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2020Learning Mesh-Based Simulation with Graph Networkscited by 840
- 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
- 2023Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstructioncited by 753
- 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
- 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 742
- 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
- 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
- 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
- 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
- 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
- 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
- 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
- 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 368
- 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
- 2023DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenescited by 359
- 2023Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesiscited by 341
- 2023SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenescited by 339
- 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
- 2018Learning 3D Human Dynamics from Videocited by 315
- 2022Flexible Diffusion Modeling of Long Videoscited by 294
50 most cited of 136 All 136 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.