# 3D-LMNet: Latent Embedding Matching for Accurate and Diverse 3D Point Cloud Reconstruction from a Single Image

arXiv: https://arxiv.org/abs/1807.07796 · HTML: https://mlsanity.com/paper/1807.07796/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2018.

## Built on by (most cited first, 30 of 40)

- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [LRM: Large Reconstruction Model for Single Image to 3D](https://mlsanity.com/paper/2311.04400/) (2023)
- [AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation](https://mlsanity.com/paper/2203.09516/) (2022)
- [POCO: Point Convolution for Surface Reconstruction](https://mlsanity.com/paper/2201.01831/) (2022)
- [CLIP goes 3D: Leveraging Prompt Tuning for Language Grounded 3D Recognition](https://mlsanity.com/paper/2303.11313/) (2023)
- [NAVI: Category-Agnostic Image Collections with High-Quality 3D Shape and Pose Annotations](https://mlsanity.com/paper/2306.09109/) (2023)
- [DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Image](https://mlsanity.com/paper/2311.18610/) (2023)
- [Reconstructing Hand-Held Objects from Monocular Video](https://mlsanity.com/paper/2211.16835/) (2022)
- [NeAT: Learning Neural Implicit Surfaces with Arbitrary Topologies from Multi-view Images](https://mlsanity.com/paper/2303.12012/) (2023)
- [3DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Process](https://mlsanity.com/paper/2303.10406/) (2023)
- [TetSphere Splatting: Representing High-Quality Geometry with Lagrangian Volumetric Meshes](https://mlsanity.com/paper/2405.20283/) (2024)
- [3D-RETR: End-to-End Single and Multi-View 3D Reconstruction with Transformers](https://mlsanity.com/paper/2110.08861/) (2021)
- [4D Myocardium Reconstruction with Decoupled Motion and Shape Model](https://mlsanity.com/paper/2308.14083/) (2023)
- [Local Aggressive Adversarial Attacks on 3D Point Cloud](https://mlsanity.com/paper/2105.09090/) (2021)
- [NU-MCC: Multiview Compressive Coding with Neighborhood Decoder and Repulsive UDF](https://mlsanity.com/paper/2307.09112/) (2023)
- [StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimation](https://mlsanity.com/paper/2506.20756/) (2025)
- [Flow-based GAN for 3D Point Cloud Generation from a Single Image](https://mlsanity.com/paper/2210.04072/) (2022)
- [Synthesizing realistic sand assemblies with denoising diffusion in latent space](https://mlsanity.com/paper/2306.04411/) (2023)
- [TreeSBA: Tree-Transformer for Self-Supervised Sequential Brick Assembly](https://mlsanity.com/paper/2407.15648/) (2024)
- [Visual Enhanced 3D Point Cloud Reconstruction from A Single Image](https://mlsanity.com/paper/2108.07685/) (2021)
- [FastMESH: Fast Surface Reconstruction by Hexagonal Mesh-based Neural Rendering](https://mlsanity.com/paper/2305.17858/) (2023)
- [Neural Geometry Image-Based Representations with Optimal Transport (OT)](https://mlsanity.com/paper/2511.18679/) (2025)
- [Inverse Rendering for High-Genus Surface Meshes from Multi-View Images](https://mlsanity.com/paper/2511.18680/) (2025)
- [Look, Cast and Mold: Learning 3D Shape Manifold from Single-view Synthetic Data](https://mlsanity.com/paper/2103.04789/) (2021)
- [D-OccNet: Detailed 3D Reconstruction Using Cross-Domain Learning](https://mlsanity.com/paper/2104.13854/) (2021)
- [A Generative Neural Network Approach for 3D Multi-Criteria Design Generation and Optimization of an Engine Mount for an Unmanned Air Vehicle](https://mlsanity.com/paper/2311.03414/) (2023)
- [PT43D: A Probabilistic Transformer for Generating 3D Shapes from Single Highly-Ambiguous RGB Images](https://mlsanity.com/paper/2405.11914/) (2024)
- [ConcreTizer: Model Inversion Attack via Occupancy Classification and Dispersion Control for 3D Point Cloud Restoration](https://mlsanity.com/paper/2503.06986/) (2025)
- [Neural Implicit 3D Shapes from Single Images with Spatial Patterns](https://mlsanity.com/paper/2106.03087/) (2021)

## Abstract (arXiv)

3D reconstruction from single view images is an ill-posed problem. Inferring the hidden regions from self-occluded images is both challenging and ambiguous. We propose a two-pronged approach to address these issues. To better incorporate the data prior and generate meaningful reconstructions, we propose 3D-LMNet, a latent embedding matching approach for 3D reconstruction. We first train a 3D point cloud auto-encoder and then learn a mapping from the 2D image to the corresponding learnt embedding. To tackle the issue of uncertainty in the reconstruction, we predict multiple reconstructions that are consistent with the input view. This is achieved by learning a probablistic latent space with a novel view-specific diversity loss. Thorough quantitative and qualitative analysis is performed to highlight the significance of the proposed approach. We outperform state-of-the-art approaches on the task of single-view 3D reconstruction on both real and synthetic datasets while generating multiple plausible reconstructions, demonstrating the generalizability and utility of our approach.