# Deep Polycuboid Fitting for Compact 3D Representation of Indoor Scenes

arXiv: https://arxiv.org/abs/2503.14912 · HTML: https://mlsanity.com/paper/2503.14912/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 19 Mar 2025.

## Builds on (most cited first, 20 of 20)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation](https://mlsanity.com/paper/2004.01658/) (2020)
- [Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions](https://mlsanity.com/paper/2004.03967/) (2020)
- [SoftGroup for 3D Instance Segmentation on Point Clouds](https://mlsanity.com/paper/2203.01509/) (2022)
- [Learning Shape Abstractions by Assembling Volumetric Primitives](https://mlsanity.com/paper/1612.00404/) (2016)
- [Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids](https://mlsanity.com/paper/1904.09970/) (2019)
- [ParSeNet: A Parametric Surface Fitting Network for 3D Point Clouds](https://mlsanity.com/paper/2003.12181/) (2020)
- [Supervised Fitting of Geometric Primitives to 3D Point Clouds](https://mlsanity.com/paper/1811.08988/) (2018)
- [Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds](https://mlsanity.com/paper/2106.03437/) (2021)
- [HPNet: Deep Primitive Segmentation Using Hybrid Representations](https://mlsanity.com/paper/2105.10620/) (2021)
- [Learning to Reconstruct 3D Manhattan Wireframes from a Single Image](https://mlsanity.com/paper/1905.07482/) (2019)
- [Robust and Accurate Superquadric Recovery: a Probabilistic Approach](https://mlsanity.com/paper/2111.14517/) (2021)
- [Primitive-based Shape Abstraction via Nonparametric Bayesian Inference](https://mlsanity.com/paper/2203.14714/) (2022)
- [DODA: Data-oriented Sim-to-Real Domain Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2204.01599/) (2022)
- [MonteBoxFinder: Detecting and Filtering Primitives to Fit a Noisy Point Cloud](https://mlsanity.com/paper/2207.14268/) (2022)
- [Efficient Path Planning in Narrow Passages for Robots with Ellipsoidal Components](https://mlsanity.com/paper/2104.04658/) (2021)
- [Efficient Plane-Based Optimization of Geometry and Texture for Indoor RGB-D Reconstruction](https://mlsanity.com/paper/1905.08853/) (2019)

## Abstract (arXiv)

This paper presents a novel framework for compactly representing a 3D indoor scene using a set of polycuboids through a deep learning-based fitting method. Indoor scenes mainly consist of man-made objects, such as furniture, which often exhibit rectilinear geometry. This property allows indoor scenes to be represented using combinations of polycuboids, providing a compact representation that benefits downstream applications like furniture rearrangement. Our framework takes a noisy point cloud as input and first detects six types of cuboid faces using a transformer network. Then, a graph neural network is used to validate the spatial relationships of the detected faces to form potential polycuboids. Finally, each polycuboid instance is reconstructed by forming a set of boxes based on the aggregated face labels. To train our networks, we introduce a synthetic dataset encompassing a diverse range of cuboid and polycuboid shapes that reflect the characteristics of indoor scenes. Our framework generalizes well to real-world indoor scene datasets, including Replica, ScanNet, and scenes captured with an iPhone. The versatility of our method is demonstrated through practical applications, such as virtual room tours and scene editing.