# Deformation-Aware 3D Model Embedding and Retrieval

arXiv: https://arxiv.org/abs/2004.01228 · HTML: https://mlsanity.com/paper/2004.01228/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2020.

## Built on by (most cited first, 30 of 30)

- [CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields](https://mlsanity.com/paper/2112.05139/) (2021)
- [Point2Cyl: Reverse Engineering 3D Objects from Point Clouds to Extrusion Cylinders](https://mlsanity.com/paper/2112.09329/) (2021)
- [Joint Learning of 3D Shape Retrieval and Deformation](https://mlsanity.com/paper/2101.07889/) (2021)
- [Reconstructing editable prismatic CAD from rounded voxel models](https://mlsanity.com/paper/2209.01161/) (2022)
- [Img2CAD: Reverse Engineering 3D CAD Models from Images through VLM-Assisted Conditional Factorization](https://mlsanity.com/paper/2408.01437/) (2024)
- [Neural Jacobian Fields: Learning Intrinsic Mappings of Arbitrary Meshes](https://mlsanity.com/paper/2205.02904/) (2022)
- [CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Sensor Simulation](https://mlsanity.com/paper/2311.01447/) (2023)
- [Edit-DiffNeRF: Editing 3D Neural Radiance Fields using 2D Diffusion Model](https://mlsanity.com/paper/2306.09551/) (2023)
- [Objaverse++: Curated 3D Object Dataset with Quality Annotations](https://mlsanity.com/paper/2504.07334/) (2025)
- [Back to Reality: Weakly-supervised 3D Object Detection with Shape-guided Label Enhancement](https://mlsanity.com/paper/2203.05238/) (2022)
- [U-RED: Unsupervised 3D Shape Retrieval and Deformation for Partial Point Clouds](https://mlsanity.com/paper/2308.06383/) (2023)
- [DPF-Net: Combining Explicit Shape Priors in Deformable Primitive Field for Unsupervised Structural Reconstruction of 3D Objects](https://mlsanity.com/paper/2308.13225/) (2023)
- [ShapeMatcher: Self-Supervised Joint Shape Canonicalization, Segmentation, Retrieval and Deformation](https://mlsanity.com/paper/2311.11106/) (2023)
- [PatchRD: Detail-Preserving Shape Completion by Learning Patch Retrieval and Deformation](https://mlsanity.com/paper/2207.11790/) (2022)
- [Structure-Aware 3D VR Sketch to 3D Shape Retrieval](https://mlsanity.com/paper/2209.09043/) (2022)
- [UrbanCAD: Towards Highly Controllable and Photorealistic 3D Vehicles for Urban Scene Simulation](https://mlsanity.com/paper/2411.19292/) (2024)
- [KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformation](https://mlsanity.com/paper/2403.10099/) (2024)
- [CORSAIR: Convolutional Object Retrieval and Symmetry-AIded Registration](https://mlsanity.com/paper/2103.06911/) (2021)
- [Fine-Grained VR Sketching: Dataset and Insights](https://mlsanity.com/paper/2209.10008/) (2022)
- [Neural Volumetric Mesh Generator](https://mlsanity.com/paper/2210.03158/) (2022)
- [Fast Point Cloud to Mesh Reconstruction for Deformable Object Tracking](https://mlsanity.com/paper/2311.02749/) (2023)
- [Towards 3D VR-Sketch to 3D Shape Retrieval](https://mlsanity.com/paper/2209.10020/) (2022)
- [A Review of Deep Learning-Powered Mesh Reconstruction Methods](https://mlsanity.com/paper/2303.02879/) (2023)
- [MADrive: Memory-Augmented Driving Scene Modeling](https://mlsanity.com/paper/2506.21520/) (2025)
- [Weakly-Supervised End-to-End CAD Retrieval to Scan Objects](https://mlsanity.com/paper/2203.12873/) (2022)
- [Fabrication-Aware Reverse Engineering for Carpentry](https://mlsanity.com/paper/2107.09965/) (2021)
- [Bent & Broken Bicycles: Leveraging synthetic data for damaged object re-identification](https://mlsanity.com/paper/2304.07883/) (2023)
- [HybridWorldSim: A Scalable and Controllable High-fidelity Simulator for Autonomous Driving](https://mlsanity.com/paper/2511.22187/) (2025)
- [Using Adaptive Gradient for Texture Learning in Single-View 3D Reconstruction](https://mlsanity.com/paper/2104.14169/) (2021)
- [PQDT: Pseudo-Query Dual Transformer for Robust Point Cloud Restoration](https://mlsanity.com/paper/2605.25127/) (2026)

## Abstract (arXiv)

We introduce a new problem of retrieving 3D models that are deformable to a given query shape and present a novel deep deformation-aware embedding to solve this retrieval task. 3D model retrieval is a fundamental operation for recovering a clean and complete 3D model from a noisy and partial 3D scan. However, given a finite collection of 3D shapes, even the closest model to a query may not be satisfactory. This motivates us to apply 3D model deformation techniques to adapt the retrieved model so as to better fit the query. Yet, certain restrictions are enforced in most 3D deformation techniques to preserve important features of the original model that prevent a perfect fitting of the deformed model to the query. This gap between the deformed model and the query induces asymmetric relationships among the models, which cannot be handled by typical metric learning techniques. Thus, to retrieve the best models for fitting, we propose a novel deep embedding approach that learns the asymmetric relationships by leveraging location-dependent egocentric distance fields. We also propose two strategies for training the embedding network. We demonstrate that both of these approaches outperform other baselines in our experiments with both synthetic and real data. Our project page can be found at https://deformscan2cad.github.io/.