# Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding

arXiv: https://arxiv.org/abs/1902.09777 · HTML: https://mlsanity.com/paper/1902.09777/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2019.

## Built on by (most cited first, 30 of 56)

- [MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction](https://mlsanity.com/paper/2206.00665/) (2022)
- [P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior](https://mlsanity.com/paper/2204.02091/) (2022)
- [PC2: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction](https://mlsanity.com/paper/2302.10668/) (2023)
- [iDisc: Internal Discretization for Monocular Depth Estimation](https://mlsanity.com/paper/2304.06334/) (2023)
- [Planar Surface Reconstruction from Sparse Views](https://mlsanity.com/paper/2103.14644/) (2021)
- [HPNet: Deep Primitive Segmentation Using Hybrid Representations](https://mlsanity.com/paper/2105.10620/) (2021)
- [StructDepth: Leveraging the structural regularities for self-supervised indoor depth estimation](https://mlsanity.com/paper/2108.08574/) (2021)
- [Understanding 3D Object Articulation in Internet Videos](https://mlsanity.com/paper/2203.16531/) (2022)
- [PlaneTR: Structure-Guided Transformers for 3D Plane Recovery](https://mlsanity.com/paper/2107.13108/) (2021)
- [PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2203.12082/) (2022)
- [PlanarRecon: Real-time 3D Plane Detection and Reconstruction from Posed Monocular Videos](https://mlsanity.com/paper/2206.07710/) (2022)
- [PlaneFormers: From Sparse View Planes to 3D Reconstruction](https://mlsanity.com/paper/2208.04307/) (2022)
- [NeurMiPs: Neural Mixture of Planar Experts for View Synthesis](https://mlsanity.com/paper/2204.13696/) (2022)
- [NOPE-SAC: Neural One-Plane RANSAC for Sparse-View Planar 3D Reconstruction](https://mlsanity.com/paper/2211.16799/) (2022)
- [Plan2Scene: Converting Floorplans to 3D Scenes](https://mlsanity.com/paper/2106.05375/) (2021)
- [Dataset Condensation via Generative Model](https://mlsanity.com/paper/2309.07698/) (2023)
- [Mirror3D: Depth Refinement for Mirror Surfaces](https://mlsanity.com/paper/2106.06629/) (2021)
- [Geometrically-driven Aggregation for Zero-shot 3D Point Cloud Understanding](https://mlsanity.com/paper/2312.02244/) (2023)
- [AirPlanes: Accurate Plane Estimation via 3D-Consistent Embeddings](https://mlsanity.com/paper/2406.08960/) (2024)
- [PlanarSplatting: Accurate Planar Surface Reconstruction in 3 Minutes](https://mlsanity.com/paper/2412.03451/) (2024)
- [PLNet: Plane and Line Priors for Unsupervised Indoor Depth Estimation](https://mlsanity.com/paper/2110.05839/) (2021)
- [Structural Multiplane Image: Bridging Neural View Synthesis and 3D Reconstruction](https://mlsanity.com/paper/2303.05937/) (2023)
- [Indoor Panorama Planar 3D Reconstruction via Divide and Conquer](https://mlsanity.com/paper/2106.14166/) (2021)
- [PlaneRecNet: Multi-Task Learning with Cross-Task Consistency for Piece-Wise Plane Detection and Reconstruction from a Single RGB Image](https://mlsanity.com/paper/2110.11219/) (2021)
- [PlaneSegNet: Fast and Robust Plane Estimation Using a Single-stage Instance Segmentation CNN](https://mlsanity.com/paper/2103.15428/) (2021)
- [Learning to Reconstruct 3D Non-Cuboid Room Layout from a Single RGB Image](https://mlsanity.com/paper/2104.07986/) (2021)
- [Structured Outdoor Architecture Reconstruction by Exploration and Classification](https://mlsanity.com/paper/2108.07990/) (2021)
- [RoSI: Recovering 3D Shape Interiors from Few Articulation Images](https://mlsanity.com/paper/2304.06342/) (2023)
- [Polygon Detection for Room Layout Estimation using Heterogeneous Graphs and Wireframes](https://mlsanity.com/paper/2306.12203/) (2023)
- [PlaneRecTR++: Unified Query Learning for Joint 3D Planar Reconstruction and Pose Estimation](https://mlsanity.com/paper/2307.13756/) (2023)

## Abstract (arXiv)

Single-image piece-wise planar 3D reconstruction aims to simultaneously segment plane instances and recover 3D plane parameters from an image. Most recent approaches leverage convolutional neural networks (CNNs) and achieve promising results. However, these methods are limited to detecting a fixed number of planes with certain learned order. To tackle this problem, we propose a novel two-stage method based on associative embedding, inspired by its recent success in instance segmentation. In the first stage, we train a CNN to map each pixel to an embedding space where pixels from the same plane instance have similar embeddings. Then, the plane instances are obtained by grouping the embedding vectors in planar regions via an efficient mean shift clustering algorithm. In the second stage, we estimate the parameter for each plane instance by considering both pixel-level and instance-level consistencies. With the proposed method, we are able to detect an arbitrary number of planes. Extensive experiments on public datasets validate the effectiveness and efficiency of our method. Furthermore, our method runs at 30 fps at the testing time, thus could facilitate many real-time applications such as visual SLAM and human-robot interaction. Code is available at https://github.com/svip-lab/PlanarReconstruction.