mlsanityData to 7 Oct 2026

CAT3D: Create Anything in 3D with Multi-View Diffusion Models

arXiv 2405.10314 ↗First posted 16 May 2024cs.CVcited by 425

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Advances in 3D reconstruction have enabled high-quality 3D capture, but require a user to collect hundreds to thousands of images to create a 3D scene. We present CAT3D, a method for creating anything in 3D by simulating this real-world capture process with a multi-view diffusion model. Given any number of input images and a set of target novel viewpoints, our model generates highly consistent novel views of a scene. These generated views can be used as input to robust 3D reconstruction techniques to produce 3D representations that can be rendered from any viewpoint in real-time. CAT3D can create entire 3D scenes in as little as one minute, and outperforms existing methods for single image and few-view 3D scene creation. See our project page for results and interactive demos at https://cat3d.github.io .

Key facts. Cited by 425 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 65 of its 76 references that are in the map. First posted on arXiv 16 May 2024.

Builds on

65 of 76 references.

Closest ancestors · 21

Same direction, newest first

  1. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation2 Dec
  2. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion14 Nov
  3. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model10 Nov
  4. 2023ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image27 Oct
  5. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior25 Oct
  6. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  7. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  8. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  9. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
  10. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  11. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Data13 Jun
  12. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
9 more
  1. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
  2. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
  3. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  4. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  5. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generation9 Mar
  6. 2023Monocular Depth Estimation using Diffusion Models28 Feb
  7. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  8. 2022Novel View Synthesis with Diffusion Models6 Oct
  9. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  5. 2022Classifier-Free Diffusion Guidancecited by 5,717
  6. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  7. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  8. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  10. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  11. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  12. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
32 more
  1. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  2. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  3. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  4. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  5. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  6. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  7. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 667
  8. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  9. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  10. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 482
  11. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  12. 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 413
  13. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  14. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 344
  15. 2023FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularizationcited by 310
  16. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  17. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  18. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  19. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  20. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 222
  21. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
  22. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 179
  23. 2023WonderJourney: Going from Anywhere to Everywherecited by 157
  24. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  25. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  26. 2024Video Interpolation with Diffusion Modelscited by 67
  27. 2023ConsistNet: Enforcing 3D Consistency for Multi-view Images Diffusioncited by 65
  28. 2023NeRFiller: Completing Scenes via Generative 3D Inpaintingcited by 62
  29. 2023Shadows Don't Lie and Lines Can't Bend! Generative Models don't know Projective Geometry...for nowcited by 61
  30. 2023SimpleNeRF: Regularizing Sparse Input Neural Radiance Fields with Simpler Solutionscited by 44
  31. 2024Disentangled 3D Scene Generation with Layout Learningcited by 33
  32. 2023SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identitycited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 93; 2025: 193; 2026: 139.

425 papers.

Sort:

By area:

  1. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  2. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  3. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  4. 2024No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Imagescited by 241
  5. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
  6. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  7. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
  8. 2024LVSM: A Large View Synthesis Model with Minimal 3D Inductive Biascited by 178
  9. 2025History-Guided Video Diffusioncited by 165
  10. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  11. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  12. 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 160
38 more
  1. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  2. 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 152
  3. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  4. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  5. 2024ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Modelcited by 139
  6. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 139
  7. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
  8. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 120
  9. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 113
  10. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Viewscited by 111
  11. 2024MV-Adapter: Multi-view Consistent Image Generation Made Easycited by 111
  12. 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
  13. 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 101
  14. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 98
  15. 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Modelscited by 97
  16. 2025Aether: Geometric-Aware Unified World Modelingcited by 94
  17. 2025Cameras as Relative Positional Encodingcited by 94
  18. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 93
  19. 2024SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpointscited by 92
  20. 2025MEt3R: Measuring Multi-View Consistency in Generated Imagescited by 91
  21. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
  22. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  23. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 77
  24. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
  25. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 68
  26. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  27. 2024SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Drivingcited by 66
  28. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 66
  29. 2024Controlling Space and Time with Diffusion Modelscited by 65
  30. 2024SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglementcited by 65
  31. 2024You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scalecited by 65
  32. 2024World-consistent Video Diffusion with Explicit 3D Modelingcited by 62
  33. 2024PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Modelscited by 60
  34. 2025Bolt3D: Generating 3D Scenes in Secondscited by 57
  35. 2024MeshFormer: High-Quality Mesh Generation with 3D-Guided Reconstruction Modelcited by 55
  36. 2024LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generationcited by 53
  37. 2024InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Modelscited by 53
  38. 2024STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenescited by 53

50 most cited of 425 All 425 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior
  2. 2024Diffusion Models in 3D Vision: A Survey
  3. 2024V3D: Video Diffusion Models are Effective 3D Generators
  4. 2024Cycle3D: High-quality and Consistent Image-to-3D Generation via Generation-Reconstruction Cycle
  5. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation
  6. 20243D-Adapter: Geometry-Consistent Multi-View Diffusion for High-Quality 3D Generation
  7. 2024Hi3D: Pursuing High-Resolution Image-to-3D Generation with Video Diffusion Models
  8. 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image