# UniPlane: Unified Plane Detection and Reconstruction from Posed Monocular Videos

arXiv: https://arxiv.org/abs/2407.03594 · HTML: https://mlsanity.com/paper/2407.03594/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 4 Jul 2024.

## Builds on (most cited first, 25 of 25)

- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans](https://mlsanity.com/paper/1812.07003/) (2018)
- [Atlas: End-to-End 3D Scene Reconstruction from Posed Images](https://mlsanity.com/paper/2003.10432/) (2020)
- [PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image](https://mlsanity.com/paper/1812.04072/) (2018)
- [PlaneNet: Piece-wise Planar Reconstruction from a Single RGB Image](https://mlsanity.com/paper/1804.06278/) (2018)
- [TorchSparse: Efficient Point Cloud Inference Engine](https://mlsanity.com/paper/2204.10319/) (2022)
- [Panoptic 3D Scene Reconstruction From a Single RGB Image](https://mlsanity.com/paper/2111.02444/) (2021)
- [Neural RGB->D Sensing: Depth and Uncertainty from a Video Camera](https://mlsanity.com/paper/1901.02571/) (2019)
- [MVDepthNet: Real-time Multiview Depth Estimation Neural Network](https://mlsanity.com/paper/1807.08563/) (2018)
- [Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks](https://mlsanity.com/paper/2011.13118/) (2020)
- [Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding](https://mlsanity.com/paper/1902.09777/) (2019)
- [Multi-view 3D Reconstruction with Transformer](https://mlsanity.com/paper/2103.12957/) (2021)
- [Multi-View Stereo by Temporal Nonparametric Fusion](https://mlsanity.com/paper/1904.06397/) (2019)
- [Planar Surface Reconstruction from Sparse Views](https://mlsanity.com/paper/2103.14644/) (2021)
- [RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction](https://mlsanity.com/paper/2011.14744/) (2020)
- [PlaneTR: Structure-Guided Transformers for 3D Plane Recovery](https://mlsanity.com/paper/2107.13108/) (2021)
- [TransMatcher: Deep Image Matching Through Transformers for Generalizable Person Re-identification](https://mlsanity.com/paper/2105.14432/) (2021)
- [RevealNet: Seeing Behind Objects in RGB-D Scans](https://mlsanity.com/paper/1904.12012/) (2019)
- [PlanarRecon: Real-time 3D Plane Detection and Reconstruction from Posed Monocular Videos](https://mlsanity.com/paper/2206.07710/) (2022)
- [Indoor Panorama Planar 3D Reconstruction via Divide and Conquer](https://mlsanity.com/paper/2106.14166/) (2021)

## Built on by (most cited first, 3 of 3)

- [Planar Gaussian Splatting](https://mlsanity.com/paper/2412.01931/) (2024)
- [PixCuboid: Room Layout Estimation from Multi-view Featuremetric Alignment](https://mlsanity.com/paper/2508.04659/) (2025)
- [GSPlane: Concise and Accurate Planar Reconstruction via Structured Representation](https://mlsanity.com/paper/2510.17095/) (2025)

## Abstract (arXiv)

We present UniPlane, a novel method that unifies plane detection and reconstruction from posed monocular videos. Unlike existing methods that detect planes from local observations and associate them across the video for the final reconstruction, UniPlane unifies both the detection and the reconstruction tasks in a single network, which allows us to directly optimize final reconstruction quality and fully leverage temporal information. Specifically, we build a Transformers-based deep neural network that jointly constructs a 3D feature volume for the environment and estimates a set of per-plane embeddings as queries. UniPlane directly reconstructs the 3D planes by taking dot products between voxel embeddings and the plane embeddings followed by binary thresholding. Extensive experiments on real-world datasets demonstrate that UniPlane outperforms state-of-the-art methods in both plane detection and reconstruction tasks, achieving +4.6 in F-score in geometry as well as consistent improvements in other geometry and segmentation metrics.