mlsanityData to 7 Oct 2026

Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Video

arXiv 2311.02848 ↗First posted 6 Nov 2023cs.CVcited by 124

Not on the map. Why?

Cited by 38 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper, we present Consistent4D, a novel approach for generating 4D dynamic objects from uncalibrated monocular videos. Uniquely, we cast the 360-degree dynamic object reconstruction as a 4D generation problem, eliminating the need for tedious multi-view data collection and camera calibration. This is achieved by leveraging the object-level 3D-aware image diffusion model as the primary supervision signal for training Dynamic Neural Radiance Fields (DyNeRF). Specifically, we propose a Cascade DyNeRF to facilitate stable convergence and temporal continuity under the supervision signal which is discrete along the time axis. To achieve spatial and temporal consistency, we further introduce an Interpolation-driven Consistency Loss. It is optimized by minimizing the discrepancy between rendered frames from DyNeRF and interpolated frames from a pre-trained video interpolation model. Extensive experiments show that our Consistent4D can perform competitively to prior art alternatives, opening up new possibilities for 4D dynamic object generation from monocular videos, whilst also demonstrating advantage for conventional text-to-3D generation tasks. Our project page is https://consistent4d.github.io/.

Key facts. Cited by 124 papers in the map (data through 7 Oct 2026). Builds on 47 of its 53 references that are in the map. First posted on arXiv 6 Nov 2023.

Builds on

47 of 53 references.

References · 47

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2023Segment Anythingcited by 8,158
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  8. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  9. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  10. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  11. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  12. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
35 more
  1. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  2. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  3. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  4. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  5. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  6. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  7. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 742
  8. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  9. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  10. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  11. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  12. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  13. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
  14. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 569
  15. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 470
  16. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  17. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  18. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  19. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  20. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  21. 20163D Menagerie: Modeling the 3D shape and pose of animalscited by 310
  22. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  23. 2022Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Renderingcited by 217
  24. 2020Real-Time Intermediate Flow Estimation for Video Frame Interpolationcited by 211
  25. 2023Text-To-4D Dynamic Scene Generationcited by 204
  26. 2022Monocular Dynamic View Synthesis: A Reality Checkcited by 199
  27. 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 195
  28. 2022TAVA: Template-free Animatable Volumetric Actorscited by 172
  29. 2022NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priorscited by 156
  30. 2022D2NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Videocited by 136
  31. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generationcited by 133
  32. 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 111
  33. 2021DOVE: Learning Deformable 3D Objects by Watching Videoscited by 61
  34. 2020Spatiotemporal Bundle Adjustment for Dynamic 3D Human Reconstruction in the Wildcited by 11
  35. 2021Class-agnostic Reconstruction of Dynamic Objects from Videoscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 35; 2025: 50; 2026: 33.

124 papers.

Sort:

By area:

  1. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
  2. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
  3. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  4. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  5. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  6. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  7. 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 143
  8. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 143
  9. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
  10. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  11. 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102
  12. 2024GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creationcited by 99
38 more
  1. 2024Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Modelscited by 99
  2. 2025TesserAct: Learning 4D Embodied World Modelscited by 90
  3. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  4. 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
  5. 2024SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfercited by 61
  6. 2024DreamMesh4D: Video-to-4D Generation with Sparse-Controlled Gaussian-Mesh Hybrid Representationcited by 58
  7. 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusioncited by 55
  8. 2024Comp4D: LLM-Guided Compositional 4D Scene Generationcited by 50
  9. 20254DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videoscited by 46
  10. 2024EG4D: Explicit Generation of 4D Object without Score Distillationcited by 45
  11. 2024Efficient4D: Fast Dynamic 3D Object Generation from a Single-view Videocited by 44
  12. 2025SV4D 2.0: Enhancing Spatio-Temporal Consistency in Multi-View Video Diffusion for High-Quality 4D Generationcited by 44
  13. 2025Puppeteer: Rig and Animate Your 3D Modelscited by 41
  14. 2024Vivid-ZOO: Multi-View Video Generation with Diffusion Modelcited by 40
  15. 2025Motion Anything: Any to Motion Generationcited by 40
  16. 2024DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenescited by 37
  17. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 37
  18. 2025Gaussian Variation Field Diffusion for High-fidelity Video-to-4D Synthesiscited by 33
  19. 2025AnimateAnyMesh: A Feed-Forward 4D Foundation Model for Text-Driven Universal Mesh Animationcited by 31
  20. 20254DNeX: Feed-Forward 4D Generative Modeling Made Easycited by 31
  21. 2024A Comprehensive Survey on 3D Content Generationcited by 28
  22. 20244Real-Video: Learning Generalizable Photo-Realistic 4D Video Diffusioncited by 27
  23. 2024AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animationcited by 25
  24. 2024Diffusion2: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Modelscited by 24
  25. 2024Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attentioncited by 24
  26. 2024Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perceptioncited by 23
  27. 2024Phy124: Fast Physics-Driven 4D Content Generation from a Single Imagecited by 22
  28. 2025ShapeGen4D: Towards High Quality 4D Shape Generation from Videoscited by 20
  29. 2024Trans4D: Realistic Geometry-Aware Transition for Compositional Text-to-4D Synthesiscited by 19
  30. 2024MagicPose4D: Crafting Articulated Models with Appearance and Motion Controlcited by 18
  31. 2025Compositional Generative Model of Unbounded 4D Citiescited by 16
  32. 2024PLA4D: Pixel-Level Alignments for Text-to-4D Gaussian Splattingcited by 15
  33. 20244Dynamic: Text-to-4D Generation with Hybrid Priorscited by 15
  34. 20244D Gaussian Splatting: Modeling Dynamic Scenes with Native 4D Primitivescited by 15
  35. 2025AnimaX: Animating the Inanimate in 3D with Joint Video-Pose Diffusion Modelscited by 15
  36. 2026Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesiscited by 15
  37. 2026ActionMesh: Animated 3D Mesh Generation with Temporal 3D Diffusioncited by 15
  38. 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14

50 most cited of 124 All 124 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Free4D: Tuning-free 4D Scene Generation with Spatial-Temporal Consistency
  2. 2023Make-It-4D: Synthesizing a Consistent Long-Term Dynamic Scene Video from a Single Image
  3. 2024Diffusion Priors for Dynamic View Synthesis from Monocular Videos
  4. 2024Bringing Objects to Life: training-free 4D generation from 3D objects through view consistent noise
  5. 2025Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Models
  6. 20244Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models
  7. 2026Beyond Pixels: From Video Priors to 4D Worlds
  8. 2024Vid3D: Synthesis of Dynamic 3D Scenes using 2D Video Diffusion