# PlankAssembly: Robust 3D Reconstruction from Three Orthographic Views with Learnt Shape Programs

arXiv: https://arxiv.org/abs/2308.05744 · HTML: https://mlsanity.com/paper/2308.05744/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 10 Aug 2023.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [ABC: A Big CAD Model Dataset For Geometric Deep Learning](https://mlsanity.com/paper/1812.06216/) (2018)
- [DeepCAD: A Deep Generative Network for Computer-Aided Design Models](https://mlsanity.com/paper/2105.09492/) (2021)
- [PolyGen: An Autoregressive Generative Model of 3D Meshes](https://mlsanity.com/paper/2002.10880/) (2020)
- [Atlas: End-to-End 3D Scene Reconstruction from Posed Images](https://mlsanity.com/paper/2003.10432/) (2020)
- [SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks](https://mlsanity.com/paper/2207.04632/) (2022)
- [Computer-Aided Design as Language](https://mlsanity.com/paper/2105.02769/) (2021)
- [SolidGen: An Autoregressive Model for Direct B-rep Synthesis](https://mlsanity.com/paper/2203.13944/) (2022)
- [Vitruvion: A Generative Model of Parametric CAD Sketches](https://mlsanity.com/paper/2109.14124/) (2021)
- [Engineering Sketch Generation for Computer-Aided Design](https://mlsanity.com/paper/2104.09621/) (2021)
- [Fusion 360 Gallery: A Dataset and Environment for Programmatic CAD Construction from Human Design Sequences](https://mlsanity.com/paper/2010.02392/) (2020)
- [FloorPlanCAD: A Large-Scale CAD Drawing Dataset for Panoptic Symbol Spotting](https://mlsanity.com/paper/2105.07147/) (2021)
- [GAT-CADNet: Graph Attention Network for Panoptic Symbol Spotting in CAD Drawings](https://mlsanity.com/paper/2201.00625/) (2022)
- [SPARE3D: A Dataset for SPAtial REasoning on Three-View Line Drawings](https://mlsanity.com/paper/2003.14034/) (2020)

## Built on by (most cited first, 8 of 8)

- [From 2D CAD Drawings to 3D Parametric Models: A Vision-Language Approach](https://mlsanity.com/paper/2412.11892/) (2024)
- [Drawing2CAD: Sequence-to-Sequence Learning for CAD Generation from Vector Drawings](https://mlsanity.com/paper/2508.18733/) (2025)
- [ShapeLib: Designing a library of programmatic 3D shape abstractions with Large Language Models](https://mlsanity.com/paper/2502.08884/) (2025)
- [Voxify3D: Pixel Art Meets Volumetric Rendering](https://mlsanity.com/paper/2512.07834/) (2025)
- [Learning to Infer Generative Template Programs for Visual Concepts](https://mlsanity.com/paper/2403.15476/) (2024)
- [GaussianCAD: Robust Self-Supervised CAD Reconstruction from Three Orthographic Views Using 3D Gaussian Splatting](https://mlsanity.com/paper/2503.05161/) (2025)
- [SOV-CAD: Stepwise Orthographic Views Guided CAD Modeling Sequence Reconstruction](https://mlsanity.com/paper/2607.04119/) (2026)
- [CompoSE: Compositional Synthesis and Editing of 3D Shapes via Part-Aware Control](https://mlsanity.com/paper/2605.19350/) (2026)

## Abstract (arXiv)

In this paper, we develop a new method to automatically convert 2D line drawings from three orthographic views into 3D CAD models. Existing methods for this problem reconstruct 3D models by back-projecting the 2D observations into 3D space while maintaining explicit correspondence between the input and output. Such methods are sensitive to errors and noises in the input, thus often fail in practice where the input drawings created by human designers are imperfect. To overcome this difficulty, we leverage the attention mechanism in a Transformer-based sequence generation model to learn flexible mappings between the input and output. Further, we design shape programs which are suitable for generating the objects of interest to boost the reconstruction accuracy and facilitate CAD modeling applications. Experiments on a new benchmark dataset show that our method significantly outperforms existing ones when the inputs are noisy or incomplete.