Learning a Predictable and Generative Vector Representation for Objects
Not on the map. Why?
Abstract (arXiv)
What is a good vector representation of an object? We believe that it should be generative in 3D, in the sense that it can produce new 3D objects; as well as be predictable from 2D, in the sense that it can be perceived from 2D images. We propose a novel architecture, called the TL-embedding network, to learn an embedding space with these properties. The network consists of two components: (a) an autoencoder that ensures the representation is generative; and (b) a convolutional network that ensures the representation is predictable. This enables tackling a number of tasks including voxel prediction from 2D images and 3D model retrieval. Extensive experimental analysis demonstrates the usefulness and versatility of this embedding.
Key facts. Cited by 173 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Mar 2016.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 49; 2022: 40; 2023: 47; 2024: 23; 2025: 8; 2026: 6.
173 papers.
Sort:
By area:
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
- 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
- 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 560
- 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 458
- 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
- 2021PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformerscited by 286
- 2021DeepCAD: A Deep Generative Network for Computer-Aided Design Modelscited by 271
- 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 258
- 2025SAM 3D: 3Dfy Anything in Imagescited by 247
- 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
- 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
38 more
- 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
- 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 207
- 2021NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wildcited by 136
- 2022Neural Wavelet-domain Diffusion for 3D Shape Generationcited by 132
- 2021HDR-NeRF: High Dynamic Range Neural Radiance Fieldscited by 114
- 2023NeRF-DS: Neural Radiance Fields for Dynamic Specular Objectscited by 111
- 2023Neural Kernel Surface Reconstructioncited by 104
- 2022What's in your hands? 3D Reconstruction of Generic Objects in Handscited by 90
- 2023NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Spacecited by 90
- 2022Deforming Radiance Fields with Cagescited by 78
- 2023UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generationcited by 78
- 2025RayZer: A Self-supervised Large View Synthesis Modelcited by 77
- 2023Multiview Compressive Coding for 3D Reconstructioncited by 71
- 2022Towards Implicit Text-Guided 3D Shape Generationcited by 68
- 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
- 2023AdaPoinTr: Diverse Point Cloud Completion with Adaptive Geometry-Aware Transformerscited by 67
- 2021Shelf-Supervised Mesh Prediction in the Wildcited by 65
- 2023Deep Learning on Implicit Neural Representations of Shapescited by 63
- 2021DOVE: Learning Deformable 3D Objects by Watching Videoscited by 61
- 2021Neural Fields as Learnable Kernels for 3D Reconstructioncited by 58
- 2021Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketchescited by 54
- 2022Unsupervised Point Cloud Representation Learning with Deep Neural Networks: A Surveycited by 54
- 2022Few-View Object Reconstruction with Unknown Categories and Camera Posescited by 53
- 2023NAVI: Category-Agnostic Image Collections with High-Quality 3D Shape and Pose Annotationscited by 53
- 2024AGG: Amortized Generative 3D Gaussians for Single Image to 3Dcited by 53
- 2021Self-Contrastive Learning with Hard Negative Sampling for Self-supervised Point Cloud Learningcited by 52
- 2021Learning Generative Models of Textured 3D Meshes from Real-World Imagescited by 44
- 2021Planar Surface Reconstruction from Sparse Viewscited by 40
- 2022Understanding 3D Object Articulation in Internet Videoscited by 36
- 2021Multiresolution Deep Implicit Functions for 3D Shape Representationcited by 35
- 2024MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generationcited by 35
- 2021Neural Surface Mapscited by 34
- 2022Neural Template: Topology-aware Reconstruction and Disentangled Generation of 3D Meshescited by 34
- 20213D Spatial Recognition without Spatially Labeled 3Dcited by 31
- 2022Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversioncited by 30
- 2022BNV-Fusion: Dense 3D Reconstruction using Bi-level Neural Volume Fusioncited by 29
- 2022SPAGHETTI: Editing Implicit Shapes Through Part Aware Generationcited by 28
- 20223PSDF: Three-Pole Signed Distance Function for Learning Surfaces with Arbitrary Topologiescited by 27
50 most cited of 173 All 173 (JSON)
- 2026RealCAD: Towards Real-World Image-to-CAD Reconstruction under Domain Shift and Parameter Bias31 Aug 2026
- 2026Reconstructing Humans and Objects in Interaction using Large Reconstruction Models27 Aug 2026
- 2026ASV3D: Adapting Diffusion-Based Single-View 3D Reconstruction with Extra Imagery8 Aug 2026
- 20263DMPE: 3D Multi-Perspective Embedding6 Jul 2026
- 2026LATO: 3D Mesh Flow Matching with Structured TOpology Preserving LAtents6 Mar 2026
- 2026Send Less, Perceive More: Masked Quantized Point Cloud Communication for Loss-Tolerant Collaborative Perception25 Feb 2026
- 2025Radar2Shape: 3D Shape Reconstruction from High-Frequency Radar using Multiresolution Signed Distance Functions21 Nov 2025
- 2025SAM 3D: 3Dfy Anything in Images20 Nov 2025
- 2025VaseVQA-3D: Benchmarking 3D VLMs on Ancient Greek Pottery6 Oct 2025
- 2025StrucADT: Generating Structure-controlled 3D Point Clouds with Adjacency Diffusion Transformer28 Sep 2025
- 20253D Shape Generation: A Survey27 Jun 2025
- 2025RayZer: A Self-supervised Large View Synthesis Model1 May 2025
38 more
- 2025PICO: Reconstructing 3D People In Contact with Objects24 Apr 2025
- 2025MASH: Masked Anchored SpHerical Distances for 3D Shape Representation and Generation12 Apr 2025
- 20243D Mesh Editing using Masked LRMs11 Dec 2024
- 2024CAD-NeRF: Learning NeRFs from Uncalibrated Few-view Images by CAD Model Retrieval5 Nov 2024
- 2024Enhancing Performance of Point Cloud Completion Networks with Consistency Loss9 Oct 2024
- 2024Single-image coherent reconstruction of objects and humans15 Aug 2024
- 2024V-VIPE: Variational View Invariant Pose Embedding9 Jul 2024
- 2024LOOC: Localizing Organs using Occupancy Networks and Body Surface Depth Images18 Jun 2024
- 2024Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding17 Jun 2024
- 2024OpenECAD: An Efficient Visual Language Model for Editable 3D-CAD Design14 Jun 2024
- 2024VP-LLM: Text-Driven 3D Volume Completion with Large Language Models through Patchification8 Jun 2024
- 2024Gaussian Control with Hierarchical Semantic Graphs in 3D Human Recovery21 May 2024
- 2024PT43D: A Probabilistic Transformer for Generating 3D Shapes from Single Highly-Ambiguous RGB Images20 May 2024
- 2024Lightplane: Highly-Scalable Components for Neural 3D Fields30 Apr 2024
- 2024SCorP: Statistics-Informed Dense Correspondence Prediction Directly from Unsegmented Medical Images27 Apr 2024
- 20246Img-to-3D: Few-Image Large-Scale Outdoor Driving Scene Reconstruction18 Apr 2024
- 2024MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generation4 Apr 2024
- 2024PointInfinity: Resolution-Invariant Point Diffusion Models4 Apr 2024
- 2024IPoD: Implicit Field Learning with Point Diffusion for Generalizable 3D Object Reconstruction from Single RGB-D Images30 Mar 2024
- 2024Robust 3D Shape Reconstruction in Zero-Shot from a Single Image in the Wild21 Mar 2024
- 20243DFIRES: Few Image 3D REconstruction for Scenes with Hidden Surface13 Mar 2024
- 2024Bayesian Diffusion Models for 3D Shape Reconstruction11 Mar 2024
- 20242L3: Lifting Imperfect Generated 2D Images into Accurate 3D29 Jan 2024
- 2024AGG: Amortized Generative 3D Gaussians for Single Image to 3D8 Jan 2024
- 2024CRA-PCN: Point Cloud Completion with Intra- and Inter-level Cross-Resolution Transformers3 Jan 2024
- 2023LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Dataset19 Dec 2023
- 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers14 Dec 2023
- 2023UpFusion: Novel View Diffusion from Unposed Sparse View Observations11 Dec 2023
- 2023UNeR3D: Versatile and Scalable 3D RGB Point Cloud Generation from 2D Images in Unsupervised Reconstruction10 Dec 2023
- 2023Multi-View Unsupervised Image Generation with Cross Attention Guidance7 Dec 2023
- 2023D-SCo: Dual-Stream Conditional Diffusion for Monocular Hand-Held Object Reconstruction23 Nov 2023
- 2023Two-stage Synthetic Supervising and Multi-view Consistency Self-supervising based Animal 3D Reconstruction by Single Image22 Nov 2023
- 2023MetaDreamer: Efficient Text-to-3D Creation With Disentangling Geometry and Texture16 Nov 2023
- 2023Registered and Segmented Deformable Object Reconstruction from a Single View Point Cloud13 Nov 2023
- 2023LRM: Large Reconstruction Model for Single Image to 3D8 Nov 2023
- 2023UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generation2 Nov 2023
- 2023NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Space26 Sep 2023
- 2023Benchmarking Encoder-Decoder Architectures for Biplanar X-ray to 3D Shape Reconstruction24 Sep 2023
50 newest of 173 All 173 (JSON)
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
- 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 560
- 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 458
- 2024Bayesian Diffusion Models for 3D Shape Reconstructioncited by 19
- 2024PointInfinity: Resolution-Invariant Point Diffusion Modelscited by 15
- 2026LATO: 3D Mesh Flow Matching with Structured TOpology Preserving LAtentscited by 10
- 2023MetaDreamer: Efficient Text-to-3D Creation With Disentangling Geometry and Texturecited by 3
- 2023Multi-View Unsupervised Image Generation with Cross Attention Guidancecited by 1
- 2025StrucADT: Generating Structure-controlled 3D Point Clouds with Adjacency Diffusion Transformercited by 0
- 2026LATO: 3D Mesh Flow Matching with Structured TOpology Preserving LAtents6 Mar 2026
- 2025StrucADT: Generating Structure-controlled 3D Point Clouds with Adjacency Diffusion Transformer28 Sep 2025
- 2024PointInfinity: Resolution-Invariant Point Diffusion Models4 Apr 2024
- 2024Bayesian Diffusion Models for 3D Shape Reconstruction11 Mar 2024
- 2023Multi-View Unsupervised Image Generation with Cross Attention Guidance7 Dec 2023
- 2023MetaDreamer: Efficient Text-to-3D Creation With Disentangling Geometry and Texture16 Nov 2023
- 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun 2023
- 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar 2023
- 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models26 Jan 2023
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
- 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
- 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
- 2021PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformerscited by 286
- 2021DeepCAD: A Deep Generative Network for Computer-Aided Design Modelscited by 271
- 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 258
- 2025SAM 3D: 3Dfy Anything in Imagescited by 247
- 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
- 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
- 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
- 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 207
- 2021NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wildcited by 136
12 most cited of 163 All 163 (JSON)
- 2026RealCAD: Towards Real-World Image-to-CAD Reconstruction under Domain Shift and Parameter Bias31 Aug 2026
- 2026Reconstructing Humans and Objects in Interaction using Large Reconstruction Models27 Aug 2026
- 2026ASV3D: Adapting Diffusion-Based Single-View 3D Reconstruction with Extra Imagery8 Aug 2026
- 20263DMPE: 3D Multi-Perspective Embedding6 Jul 2026
- 2026Send Less, Perceive More: Masked Quantized Point Cloud Communication for Loss-Tolerant Collaborative Perception25 Feb 2026
- 2025Radar2Shape: 3D Shape Reconstruction from High-Frequency Radar using Multiresolution Signed Distance Functions21 Nov 2025
- 2025SAM 3D: 3Dfy Anything in Images20 Nov 2025
- 20253D Shape Generation: A Survey27 Jun 2025
- 2025RayZer: A Self-supervised Large View Synthesis Model1 May 2025
- 2025PICO: Reconstructing 3D People In Contact with Objects24 Apr 2025
- 2025MASH: Masked Anchored SpHerical Distances for 3D Shape Representation and Generation12 Apr 2025
- 20243D Mesh Editing using Masked LRMs11 Dec 2024
12 newest of 163 All 163 (JSON)