# Multi-Plane Program Induction with 3D Box Priors

arXiv: https://arxiv.org/abs/2011.10007 · HTML: https://mlsanity.com/paper/2011.10007/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2020.

## Built on by (most cited first, 10 of 10)

- [Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Image](https://mlsanity.com/paper/2203.09457/) (2022)
- [Can Large Language Models Understand Symbolic Graphics Programs?](https://mlsanity.com/paper/2408.08313/) (2024)
- [PlaneTR: Structure-Guided Transformers for 3D Plane Recovery](https://mlsanity.com/paper/2107.13108/) (2021)
- [HAKE: A Knowledge Engine Foundation for Human Activity Understanding](https://mlsanity.com/paper/2202.06851/) (2022)
- [Holistically-Attracted Wireframe Parsing: From Supervised to Self-Supervised Learning](https://mlsanity.com/paper/2210.12971/) (2022)
- [Test-time Adaptation with Slot-Centric Models](https://mlsanity.com/paper/2203.11194/) (2022)
- [Structure from Duplicates: Neural Inverse Graphics from a Pile of Objects](https://mlsanity.com/paper/2401.05236/) (2024)
- [Learning Continuous Implicit Representation for Near-Periodic Patterns](https://mlsanity.com/paper/2208.12278/) (2022)
- [Learning to Infer 3D Shape Programs with Differentiable Renderer](https://mlsanity.com/paper/2206.12675/) (2022)
- [ShapeCodeBench: A Renewable Benchmark for Perception-to-Program Reconstruction of Synthetic Shape Scenes](https://mlsanity.com/paper/2605.11680/) (2026)

## Abstract (arXiv)

We consider two important aspects in understanding and editing images: modeling regular, program-like texture or patterns in 2D planes, and 3D posing of these planes in the scene. Unlike prior work on image-based program synthesis, which assumes the image contains a single visible 2D plane, we present Box Program Induction (BPI), which infers a program-like scene representation that simultaneously models repeated structure on multiple 2D planes, the 3D position and orientation of the planes, and camera parameters, all from a single image. Our model assumes a box prior, i.e., that the image captures either an inner view or an outer view of a box in 3D. It uses neural networks to infer visual cues such as vanishing points, wireframe lines to guide a search-based algorithm to find the program that best explains the image. Such a holistic, structured scene representation enables 3D-aware interactive image editing operations such as inpainting missing pixels, changing camera parameters, and extrapolate the image contents.