# Planar Surface Reconstruction from Sparse Views

arXiv: https://arxiv.org/abs/2103.14644 · HTML: https://mlsanity.com/paper/2103.14644/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 72 references that are in the map.
- First posted on arXiv 26 Mar 2021.

## Builds on (most cited first, 30 of 47)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [DeepMVS: Learning Multi-view Stereopsis](https://mlsanity.com/paper/1804.00650/) (2018)
- [Learning Correspondence from the Cycle-Consistency of Time](https://mlsanity.com/paper/1903.07593/) (2019)
- [Learning to Recover 3D Scene Shape from a Single Image](https://mlsanity.com/paper/2012.09365/) (2020)
- [Working hard to know your neighbor's margins: Local descriptor learning loss](https://mlsanity.com/paper/1705.10872/) (2017)
- [DeMoN: Depth and Motion Network for Learning Monocular Stereo](https://mlsanity.com/paper/1612.02401/) (2016)
- [Learning a Multi-View Stereo Machine](https://mlsanity.com/paper/1708.05375/) (2017)
- [Neighbourhood Consensus Networks](https://mlsanity.com/paper/1810.10510/) (2018)
- [Learning a Predictable and Generative Vector Representation for Objects](https://mlsanity.com/paper/1603.08637/) (2016)
- [Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image](https://mlsanity.com/paper/2002.12212/) (2020)
- [Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters](https://mlsanity.com/paper/1904.00889/) (2019)
- [HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation](https://mlsanity.com/paper/1901.03861/) (2019)
- [PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image](https://mlsanity.com/paper/1812.04072/) (2018)
- [LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image](https://mlsanity.com/paper/1803.08999/) (2018)
- [Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views](https://mlsanity.com/paper/1505.05641/) (2015)

## Built on by (most cited first, 30 of 40)

- [PixelSynth: Generating a 3D-Consistent Experience from a Single Image](https://mlsanity.com/paper/2108.05892/) (2021)
- [The 8-Point Algorithm as an Inductive Bias for Relative Pose Prediction by ViTs](https://mlsanity.com/paper/2208.08988/) (2022)
- [Understanding 3D Object Articulation in Internet Videos](https://mlsanity.com/paper/2203.16531/) (2022)
- [Understanding 3D Object Interaction from a Single Image](https://mlsanity.com/paper/2305.09664/) (2023)
- [PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2203.12082/) (2022)
- [PlanarRecon: Real-time 3D Plane Detection and Reconstruction from Posed Monocular Videos](https://mlsanity.com/paper/2206.07710/) (2022)
- [PlaneFormers: From Sparse View Planes to 3D Reconstruction](https://mlsanity.com/paper/2208.04307/) (2022)
- [Virtual Correspondence: Humans as a Cue for Extreme-View Geometry](https://mlsanity.com/paper/2206.08365/) (2022)
- [NOPE-SAC: Neural One-Plane RANSAC for Sparse-View Planar 3D Reconstruction](https://mlsanity.com/paper/2211.16799/) (2022)
- [Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation](https://mlsanity.com/paper/2303.11329/) (2023)
- [Learning Viewpoint-Agnostic Visual Representations by Recovering Tokens in 3D Space](https://mlsanity.com/paper/2206.11895/) (2022)
- [AirPlanes: Accurate Plane Estimation via 3D-Consistent Embeddings](https://mlsanity.com/paper/2406.08960/) (2024)
- [PlanarSplatting: Accurate Planar Surface Reconstruction in 3 Minutes](https://mlsanity.com/paper/2412.03451/) (2024)
- [Structural Multiplane Image: Bridging Neural View Synthesis and 3D Reconstruction](https://mlsanity.com/paper/2303.05937/) (2023)
- [FAR: Flexible, Accurate and Robust 6DoF Relative Camera Pose Estimation](https://mlsanity.com/paper/2403.03221/) (2024)
- [What's Behind the Couch? Directed Ray Distance Functions (DRDF) for 3D Scene Reconstruction](https://mlsanity.com/paper/2112.04481/) (2021)
- [RoSI: Recovering 3D Shape Interiors from Few Articulation Images](https://mlsanity.com/paper/2304.06342/) (2023)
- [PlaneRecTR++: Unified Query Learning for Joint 3D Planar Reconstruction and Pose Estimation](https://mlsanity.com/paper/2307.13756/) (2023)
- [3DFIRES: Few Image 3D REconstruction for Scenes with Hidden Surface](https://mlsanity.com/paper/2403.08768/) (2024)
- [AlphaTablets: A Generic Plane Representation for 3D Planar Reconstruction from Monocular Videos](https://mlsanity.com/paper/2411.19950/) (2024)
- [Planar Gaussian Splatting](https://mlsanity.com/paper/2412.01931/) (2024)
- [Towards In-the-wild 3D Plane Reconstruction from a Single Image](https://mlsanity.com/paper/2506.02493/) (2025)
- [Visual Correspondence Hallucination](https://mlsanity.com/paper/2106.09711/) (2021)
- [Recognizing Scenes from Novel Viewpoints](https://mlsanity.com/paper/2112.01520/) (2021)
- [PlanarNeRF: Online Learning of Planar Primitives with Neural Radiance Fields](https://mlsanity.com/paper/2401.00871/) (2023)
- [SRPose: Two-view Relative Pose Estimation with Sparse Keypoints](https://mlsanity.com/paper/2407.08199/) (2024)
- [PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-Forward Planar Splatting](https://mlsanity.com/paper/2510.18714/) (2025)
- [HM3D-ABO: A Photo-realistic Dataset for Object-centric Multi-view 3D Reconstruction](https://mlsanity.com/paper/2206.12356/) (2022)
- [Sparse-View 3D Reconstruction: Recent Advances and Open Challenges](https://mlsanity.com/paper/2507.16406/) (2025)
- [UniPlane: Unified Plane Detection and Reconstruction from Posed Monocular Videos](https://mlsanity.com/paper/2407.03594/) (2024)

## Abstract (arXiv)

The paper studies planar surface reconstruction of indoor scenes from two views with unknown camera poses. While prior approaches have successfully created object-centric reconstructions of many scenes, they fail to exploit other structures, such as planes, which are typically the dominant components of indoor scenes. In this paper, we reconstruct planar surfaces from multiple views, while jointly estimating camera pose. Our experiments demonstrate that our method is able to advance the state of the art of reconstruction from sparse views, on challenging scenes from Matterport3D. Project site: https://jinlinyi.github.io/SparsePlanes/