mlsanityData to 7 Oct 2026

Multiview Compressive Coding for 3D Reconstruction

arXiv 2301.08247 ↗First posted 19 Jan 2023cs.CVcited by 71

Not on the map. Why?

Abstract (arXiv)

A central goal of visual recognition is to understand objects and scenes from a single image. 2D recognition has witnessed tremendous progress thanks to large-scale learning and general-purpose representations. Comparatively, 3D poses new challenges stemming from occlusions not depicted in the image. Prior works try to overcome these by inferring from multiple views or rely on scarce CAD models and category-specific priors which hinder scaling to novel settings. In this work, we explore single-view 3D reconstruction by learning generalizable representations inspired by advances in self-supervised learning. We introduce a simple framework that operates on 3D points of single objects or whole scenes coupled with category-agnostic large-scale training from diverse RGB-D videos. Our model, Multiview Compressive Coding (MCC), learns to compress the input appearance and geometry to predict the 3D structure by querying a 3D-aware decoder. MCC's generality and efficiency allow it to learn from large-scale and diverse data sources with strong generalization to novel objects imagined by DALL·E 2 or captured in-the-wild with an iPhone.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Builds on 61 of its 88 references that are in the map. First posted on arXiv 19 Jan 2023.

Builds on

61 of 88 references.

References · 61

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2016Layer Normalizationcited by 5,579
  11. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  12. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
38 more
  1. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  2. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  3. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  4. 2021Vision Transformers for Dense Predictioncited by 1,789
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  6. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  7. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  8. 2020Neural Sparse Voxel Fieldscited by 959
  9. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  10. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  11. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  12. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  13. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  14. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  15. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  16. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  17. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  18. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
  19. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  20. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  21. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  22. 2017Neural 3D Mesh Renderercited by 437
  23. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  24. 2018PCN: Point Completion Networkcited by 382
  25. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  26. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
  27. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  28. 2021PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformerscited by 284
  29. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  30. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  31. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  32. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  33. 2019Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Imagescited by 164
  34. 2020PF-Net: Point Fractal Network for 3D Point Cloud Completioncited by 146
  35. 2017MarrNet: 3D Shape Reconstruction via 2.5D Sketchescited by 108
  36. 2016Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervisioncited by 106
  37. 2022ShapeFormer: Transformer-based Shape Completion via Sparse Representationcited by 106
  38. 2019Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformationcited by 103

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 26; 2024: 29; 2025: 13; 2026: 3.

71 papers.

Sort:

By area:

  1. 2023Segment Anythingcited by 8,158
  2. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  3. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  4. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  5. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  6. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  7. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 344
  8. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  9. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 257
  10. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  11. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 227
  12. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
38 more
  1. 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 143
  2. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generationcited by 133
  3. 2023FoundPose: Unseen Object Pose Estimation with Foundation Featurescited by 109
  4. 2023Anything-3D: Towards Single-view Anything Reconstruction in the Wildcited by 91
  5. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
  6. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Modelscited by 79
  7. 2025RayZer: A Self-supervised Large View Synthesis Modelcited by 74
  8. 2024Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single Viewcited by 59
  9. 2025Bolt3D: Generating 3D Scenes in Secondscited by 57
  10. 2023PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigmcited by 56
  11. 2024AGG: Amortized Generative 3D Gaussians for Single Image to 3Dcited by 51
  12. 2024Image Sculpting: Precise Object Editing with 3D Geometry Controlcited by 45
  13. 2025SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Imagescited by 41
  14. 2024SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAEcited by 38
  15. 2024MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generationcited by 35
  16. 2024LRM-Zero: Training Large Reconstruction Models with Synthesized Datacited by 35
  17. 2023ZeroShape: Regression-based Zero-shot Shape Reconstructioncited by 30
  18. 2024Find Any Part in 3Dcited by 30
  19. 2023Neural feels with neural fields: Visuo-tactile perception for in-hand manipulationcited by 28
  20. 2024CharacterGen: Efficient 3D Character Generation from Single Images with Multi-View Pose Canonicalizationcited by 28
  21. 2024DrivingRecon: Large 4D Gaussian Reconstruction Model For Autonomous Drivingcited by 27
  22. 2024StereoDiffusion: Training-Free Stereo Image Generation Using Latent Diffusion Modelscited by 24
  23. 2024Real3D: Scaling Up Large Reconstruction Models with Real-World Imagescited by 24
  24. 2024Reconstructing Hand-Held Objects in 3D from Images and Videoscited by 23
  25. 2024From an Image to a Scene: Learning to Imagine the World from a Million 360 Videoscited by 23
  26. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 23
  27. 2024MVD2: Efficient Multiview 3D Reconstruction for Multiview Diffusioncited by 22
  28. 2023NViST: In the Wild New View Synthesis from a Single Image with Transformerscited by 19
  29. 2023Unifying (Machine) Vision via Counterfactual World Modelingcited by 18
  30. 2024Denoising Diffusion via Image-Based Renderingcited by 15
  31. 2024Lightplane: Highly-Scalable Components for Neural 3D Fieldscited by 13
  32. 2025ZeroGrasp: Zero-Shot Shape Reconstruction Enabled Robotic Graspingcited by 13
  33. 2024GaussianAnything: Interactive Point Cloud Flow Matching For 3D Object Generationcited by 12
  34. 2024Sampling 3D Gaussian Scenes in Seconds with Latent Diffusion Modelscited by 10
  35. 2024Zero-Shot Multi-Object Scene Completioncited by 9
  36. 2024GenRC: Generative 3D Room Completion from Sparse Image Collectionscited by 9
  37. 2024Shape2Scene: 3D Scene Representation Learning Through Pre-training on Shape Datacited by 8
  38. 2024Hyperbolic-constraint Point Cloud Reconstruction from Single RGB-D Imagescited by 8

50 most cited of 71 All 71 (JSON)