mlsanityData to 7 Oct 2026

Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction

arXiv 2109.00512 ↗First posted 1 Sep 2021cs.CVcited by 646

Not on the map. Why?

Cited by 93 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Traditional approaches for learning 3D object categories have been predominantly trained and evaluated on synthetic datasets due to the unavailability of real 3D-annotated category-centric data. Our main goal is to facilitate advances in this field by collecting real-world data in a magnitude similar to the existing synthetic counterparts. The principal contribution of this work is thus a large-scale dataset, called Common Objects in 3D, with real multi-view images of object categories annotated with camera poses and ground truth 3D point clouds. The dataset contains a total of 1.5 million frames from nearly 19,000 videos capturing objects from 50 MS-COCO categories and, as such, it is significantly larger than alternatives both in terms of the number of categories and objects. We exploit this new dataset to conduct one of the first large-scale "in-the-wild" evaluations of several new-view-synthesis and category-centric 3D reconstruction methods. Finally, we contribute NerFormer - a novel neural rendering method that leverages the powerful Transformer to reconstruct an object given a small number of its views. The CO3D dataset is available at https://github.com/facebookresearch/co3d .

Key facts. Cited by 646 papers in the map (data through 7 Oct 2026). Builds on 51 of its 74 references that are in the map. First posted on arXiv 1 Sep 2021.

Builds on

51 of 74 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2016Layer Normalizationcited by 5,579
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  7. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  8. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  9. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  10. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  11. 2020Neural Sparse Voxel Fieldscited by 959
  12. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
38 more
  1. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  2. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  3. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  4. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  5. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  6. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  7. 2017Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentationscited by 614
  8. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  9. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  10. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  11. 2019PointRend: Image Segmentation as Renderingcited by 439
  12. 2017Neural 3D Mesh Renderercited by 437
  13. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  14. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 389
  15. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  16. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  17. 2019Learning Shape Templates with Structured Implicit Functionscited by 242
  18. 2016Generating Images with Perceptual Similarity Metrics based on Deep Networkscited by 221
  19. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  20. 2017Learning a Multi-View Stereo Machinecited by 194
  21. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  22. 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
  23. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  24. 2016Panoptic Studio: A Massively Multiview System for Social Interaction Capturecited by 173
  25. 2017Optimizing the Latent Space of Generative Networkscited by 157
  26. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
  27. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  28. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
  29. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  30. 2020Shape and Viewpoint without Keypointscited by 92
  31. 2018Unsupervised Learning of Shape and Pose with Differentiable Point Cloudscited by 89
  32. 2020Articulation-aware Canonical Surface Mappingcited by 80
  33. 2016Unsupervised Learning of 3D Structure from Imagescited by 67
  34. 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
  35. 2016A Large Dataset of Object Scanscited by 64
  36. 2019Canonical Surface Mapping via Geometric Cycle Consistencycited by 62
  37. 2015Multi-view 3D Models from Single Images with a Convolutional Networkcited by 57
  38. 2018Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Predictioncited by 52

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 59; 2023: 140; 2024: 130; 2025: 168; 2026: 139.

646 papers.

Sort:

By area:

  1. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  2. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  3. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  4. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  5. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  6. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  7. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  8. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  9. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  10. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 553
  11. 2021Neural Fields in Visual Computing and Beyondcited by 539
  12. 2023LERF: Language Embedded Radiance Fieldscited by 538
38 more
  1. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  2. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
  3. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  4. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 444
  5. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 425
  6. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  7. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  8. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  9. 2021Advances in Neural Renderingcited by 365
  10. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  11. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  12. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 356
  13. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 324
  14. 2023ReconFusion: 3D Reconstruction with Diffusion Priorscited by 317
  15. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  16. 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 292
  17. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  18. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  19. 20243D Reconstruction with Spatial Memorycited by 269
  20. 2021Urban Radiance Fieldscited by 263
  21. 2025AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Viewscited by 234
  22. 2025SAM 3D: 3Dfy Anything in Imagescited by 230
  23. 2023Visual Geometry Grounded Deep Structure From Motioncited by 225
  24. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
  25. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  26. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  27. 2021GeoNeRF: Generalizing NeRF with Geometry Priorscited by 195
  28. 2023COLMAP-Free 3D Gaussian Splattingcited by 193
  29. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
  30. 2025FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Viewscited by 176
  31. 2024Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairscited by 171
  32. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  33. 2023PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Predictioncited by 165
  34. 2024MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motioncited by 161
  35. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  36. 2022NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)cited by 156
  37. 2023DreamEditor: Text-Driven 3D Scene Editing with Neural Fieldscited by 154
  38. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Viewscited by 148

50 most cited of 646 All 646 (JSON)