mlsanityData to 7 Oct 2026

A Cookbook of 3D Vision: Data, Learning Paradigms, and Application

arXiv 2606.04291 ↗First posted 2 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

3D vision has rapidly evolved, driven by increasingly diverse data representations, learning paradigms, and modeling strategies. Yet the field remains fragmented across representations and benchmarks, making it difficult to develop unified perspectives on efficiency, fidelity, and scalability. This work provides a data-centric taxonomy of 3D vision that connects geometric representations, datasets, learning frameworks, and applications within a single conceptual map. We begin by analysing the principal structural representations of 3D data--point clouds, meshes, voxels, and 3D Gaussians--along with their acquisition pipelines. We then examine how dataset design, benchmark construction, and supervision regimes shape recent advances, spanning 2D-supervised 3D learning, implicit neural representations, and 4D world modeling. Through this integrative lens, we clarify the relationships among representations, learning paradigms, and downstream tasks in reconstruction, generation, and video modeling, offering a consolidated view of emerging trends toward balancing efficiency and fidelity and toward multimodal geometric grounding.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 96 of its 156 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

96 of 156 references.

References · 96

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  6. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  7. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  8. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  9. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  10. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  11. 2020Point Transformercited by 1,462
  12. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
38 more
  1. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  2. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  3. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  4. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  5. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  6. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  7. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  8. 2020Neural Sparse Voxel Fieldscited by 959
  9. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  10. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  11. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  12. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 801
  13. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  14. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  15. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  16. 2020PCT: Point cloud transformercited by 730
  17. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  18. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  19. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  20. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  21. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  22. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  23. 2023Point Transformer V3: Simpler, Faster, Strongercited by 579
  24. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  25. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
  26. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  27. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 487
  28. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  29. 2017Neural 3D Mesh Renderercited by 437
  30. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  31. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  32. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
  33. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  34. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  35. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  36. 2016Virtual Worlds as Proxy for Multi-Object Tracking Analysiscited by 340
  37. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 340
  38. 2018ABC: A Big CAD Model Dataset For Geometric Deep Learningcited by 339

50 most cited of 96 All 96 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Bridging the Dimensionality Gap: A Taxonomy and Survey of 2D Vision Model Adaptation for 3D Analysis
  2. 2026Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective
  3. 2024Diffusion Models in 3D Vision: A Survey
  4. 2022Deep Generative Models on 3D Representations: A Survey
  5. 2024ImageNet3D: Towards General-Purpose Object-Level 3D Understanding
  6. 2025Geometry and Perception Guided Gaussians for Multiview-consistent 3D Generation from a Single Image
  7. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models
  8. 20263D-IDE: 3D Implicit Depth Emergent