# PlanarRecon: Real-time 3D Plane Detection and Reconstruction from Posed Monocular Videos

arXiv: https://arxiv.org/abs/2206.07710 · HTML: https://mlsanity.com/paper/2206.07710/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 48 references that are in the map.
- First posted on arXiv 15 Jun 2022.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference](https://mlsanity.com/paper/1902.10556/) (2019)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [PatchmatchNet: Learned Multi-View Patchmatch Stereo](https://mlsanity.com/paper/2012.01411/) (2020)
- [Atlas: End-to-End 3D Scene Reconstruction from Posed Images](https://mlsanity.com/paper/2003.10432/) (2020)
- [NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo](https://mlsanity.com/paper/2109.01129/) (2021)
- [Cost Volume Pyramid Based Depth Inference for Multi-View Stereo](https://mlsanity.com/paper/1912.08329/) (2019)
- [Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awareness](https://mlsanity.com/paper/1911.12012/) (2019)
- [PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image](https://mlsanity.com/paper/1812.04072/) (2018)
- [PlaneNet: Piece-wise Planar Reconstruction from a Single RGB Image](https://mlsanity.com/paper/1804.06278/) (2018)
- [Neural RGB->D Sensing: Depth and Uncertainty from a Video Camera](https://mlsanity.com/paper/1901.02571/) (2019)
- [DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusion](https://mlsanity.com/paper/2012.02177/) (2020)
- [Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks](https://mlsanity.com/paper/2011.13118/) (2020)
- [Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding](https://mlsanity.com/paper/1902.09777/) (2019)
- [Multi-View Stereo by Temporal Nonparametric Fusion](https://mlsanity.com/paper/1904.06397/) (2019)
- [A General Optimization-based Framework for Local Odometry Estimation with Multiple Sensors](https://mlsanity.com/paper/1901.03638/) (2019)
- [Planar Surface Reconstruction from Sparse Views](https://mlsanity.com/paper/2103.14644/) (2021)
- [PlaneTR: Structure-Guided Transformers for 3D Plane Recovery](https://mlsanity.com/paper/2107.13108/) (2021)
- [Monocular Object and Plane SLAM in Structured Environments](https://mlsanity.com/paper/1809.03415/) (2018)
- [Pop-up SLAM: Semantic Monocular Plane SLAM for Low-texture Environments](https://mlsanity.com/paper/1703.07334/) (2017)
- [PatchMatch-RL: Deep MVS with Pixelwise Depth, Normal, and Visibility](https://mlsanity.com/paper/2108.08943/) (2021)
- [SurfaceNet+: An End-to-end 3D Neural Network for Very Sparse Multi-view Stereopsis](https://mlsanity.com/paper/2005.12690/) (2020)
- [Indoor Panorama Planar 3D Reconstruction via Divide and Conquer](https://mlsanity.com/paper/2106.14166/) (2021)

## Built on by (most cited first, 27 of 27)

- [Learning-based Multi-View Stereo: A Survey](https://mlsanity.com/paper/2408.15235/) (2024)
- [FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Models](https://mlsanity.com/paper/2308.05733/) (2023)
- [Pixel-Aligned Recurrent Queries for Multi-View 3D Object Detection](https://mlsanity.com/paper/2310.01401/) (2023)
- [AirPlanes: Accurate Plane Estimation via 3D-Consistent Embeddings](https://mlsanity.com/paper/2406.08960/) (2024)
- [PlanarSplatting: Accurate Planar Surface Reconstruction in 3 Minutes](https://mlsanity.com/paper/2412.03451/) (2024)
- [Structural Multiplane Image: Bridging Neural View Synthesis and 3D Reconstruction](https://mlsanity.com/paper/2303.05937/) (2023)
- [G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior](https://mlsanity.com/paper/2510.12099/) (2025)
- [VisFusion: Visibility-aware Online 3D Scene Reconstruction from Videos](https://mlsanity.com/paper/2304.10687/) (2023)
- [3DFIRES: Few Image 3D REconstruction for Scenes with Hidden Surface](https://mlsanity.com/paper/2403.08768/) (2024)
- [AlphaTablets: A Generic Plane Representation for 3D Planar Reconstruction from Monocular Videos](https://mlsanity.com/paper/2411.19950/) (2024)
- [Planar Gaussian Splatting](https://mlsanity.com/paper/2412.01931/) (2024)
- [Towards In-the-wild 3D Plane Reconstruction from a Single Image](https://mlsanity.com/paper/2506.02493/) (2025)
- [PARF: Primitive-Aware Radiance Fusion for Indoor Scene Novel View Synthesis](https://mlsanity.com/paper/2309.17190/) (2023)
- [PlanarNeRF: Online Learning of Planar Primitives with Neural Radiance Fields](https://mlsanity.com/paper/2401.00871/) (2023)
- [PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-Forward Planar Splatting](https://mlsanity.com/paper/2510.18714/) (2025)
- [UniPlane: Unified Plane Detection and Reconstruction from Posed Monocular Videos](https://mlsanity.com/paper/2407.03594/) (2024)
- [MonoPlane: Exploiting Monocular Geometric Cues for Generalizable 3D Plane Reconstruction](https://mlsanity.com/paper/2411.01226/) (2024)
- [PixCuboid: Room Layout Estimation from Multi-view Featuremetric Alignment](https://mlsanity.com/paper/2508.04659/) (2025)
- [3D Gaussian Flats: Hybrid 2D/3D Photometric Scene Reconstruction](https://mlsanity.com/paper/2509.16423/) (2025)
- [Cross-Dimensional Refined Learning for Real-Time 3D Visual Perception from Monocular Video](https://mlsanity.com/paper/2303.09248/) (2023)
- [Neural Mesh Fusion: Unsupervised 3D Planar Surface Understanding](https://mlsanity.com/paper/2402.16739/) (2024)
- [GSPlane: Concise and Accurate Planar Reconstruction via Structured Representation](https://mlsanity.com/paper/2510.17095/) (2025)
- [Multi-task Planar Reconstruction with Feature Warping Guidance](https://mlsanity.com/paper/2311.14981/) (2023)
- [PlanaReLoc: Camera Relocalization in 3D Planar Primitives via Region-Based Structure Matching](https://mlsanity.com/paper/2603.20818/) (2026)
- [Group3D: MLLM-Driven Semantic Grouping for Open-Vocabulary 3D Object Detection](https://mlsanity.com/paper/2603.21944/) (2026)
- [TopoGS: Planar Reconstruction via Topology-aware 3D Gaussian Splatting](https://mlsanity.com/paper/2607.16838/) (2026)
- [Pixel-wise Planarity for High-Precision Monocular Plane Segmentation](https://mlsanity.com/paper/2609.13246/) (2026)

## Abstract (arXiv)

We present PlanarRecon -- a novel framework for globally coherent detection and reconstruction of 3D planes from a posed monocular video. Unlike previous works that detect planes in 2D from a single image, PlanarRecon incrementally detects planes in 3D for each video fragment, which consists of a set of key frames, from a volumetric representation of the scene using neural networks. A learning-based tracking and fusion module is designed to merge planes from previous fragments to form a coherent global plane reconstruction. Such design allows PlanarRecon to integrate observations from multiple views within each fragment and temporal information across different ones, resulting in an accurate and coherent reconstruction of the scene abstraction with low-polygonal geometry. Experiments show that the proposed approach achieves state-of-the-art performances on the ScanNet dataset while being real-time.