# Automatic Reverse Engineering: Creating computer-aided design (CAD) models from multi-view images

arXiv: https://arxiv.org/abs/2309.13281 · HTML: https://mlsanity.com/paper/2309.13281/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 50 references that are in the map.
- First posted on arXiv 23 Sep 2023.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection](https://mlsanity.com/paper/2104.10956/) (2021)
- [DeepCAD: A Deep Generative Network for Computer-Aided Design Models](https://mlsanity.com/paper/2105.09492/) (2021)
- [PolyGen: An Autoregressive Generative Model of 3D Meshes](https://mlsanity.com/paper/2002.10880/) (2020)
- [Learning Gradient Fields for Shape Generation](https://mlsanity.com/paper/2008.06520/) (2020)
- [Volumetric and Multi-View CNNs for Object Classification on 3D Data](https://mlsanity.com/paper/1604.03265/) (2016)
- [SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization](https://mlsanity.com/paper/1912.07109/) (2019)
- [ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection](https://mlsanity.com/paper/2106.01178/) (2021)
- [Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision](https://mlsanity.com/paper/1612.00814/) (2016)
- [Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networks](https://mlsanity.com/paper/1909.00321/) (2019)
- [ParSeNet: A Parametric Surface Fitting Network for 3D Point Clouds](https://mlsanity.com/paper/2003.12181/) (2020)
- [BRepNet: A topological message passing system for solid models](https://mlsanity.com/paper/2104.00706/) (2021)
- [Unsupervised Learning of Shape and Pose with Differentiable Point Clouds](https://mlsanity.com/paper/1810.09381/) (2018)
- [Computer-Aided Design as Language](https://mlsanity.com/paper/2105.02769/) (2021)
- [SketchGen: Generating Constrained CAD Sketches](https://mlsanity.com/paper/2106.02711/) (2021)
- [PIE-NET: Parametric Inference of Point Cloud Edges](https://mlsanity.com/paper/2007.04883/) (2020)
- [Inferring CAD Modeling Sequences Using Zone Graphs](https://mlsanity.com/paper/2104.03900/) (2021)
- [Engineering Sketch Generation for Computer-Aided Design](https://mlsanity.com/paper/2104.09621/) (2021)
- [A Deeper Look at 3D Shape Classifiers](https://mlsanity.com/paper/1809.02560/) (2018)
- [LegoFormer: Transformers for Block-by-Block Multi-view 3D Reconstruction](https://mlsanity.com/paper/2106.12102/) (2021)
- [A Recurrent CNN for Automatic Detection and Classification of Coronary Artery Plaque and Stenosis in Coronary CT Angiography](https://mlsanity.com/paper/1804.04360/) (2018)

## Built on by (most cited first, 1 of 1)

- [MV2Cyl: Reconstructing 3D Extrusion Cylinders from Multi-View Images](https://mlsanity.com/paper/2406.10853/) (2024)

## Abstract (arXiv)

Generation of computer-aided design (CAD) models from multi-view images may be useful in many practical applications. To date, this problem is usually solved with an intermediate point-cloud reconstruction and involves manual work to create the final CAD models. In this contribution, we present a novel network for an automated reverse engineering task. Our network architecture combines three distinct stages: A convolutional neural network as the encoder stage, a multi-view pooling stage and a transformer-based CAD sequence generator. The model is trained and evaluated on a large number of simulated input images and extensive optimization of model architectures and hyper-parameters is performed. A proof-of-concept is demonstrated by successfully reconstructing a number of valid CAD models from simulated test image data. Various accuracy metrics are calculated and compared to a state-of-the-art point-based network. Finally, a real world test is conducted supplying the network with actual photographs of two three-dimensional test objects. It is shown that some of the capabilities of our network can be transferred to this domain, even though the training exclusively incorporates purely synthetic training data. However to date, the feasible model complexity is still limited to basic shapes.