# Pix2Poly: A Sequence Prediction Method for End-to-end Polygonal Building Footprint Extraction from Remote Sensing Imagery

arXiv: https://arxiv.org/abs/2412.07899 · HTML: https://mlsanity.com/paper/2412.07899/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 52 references that are in the map.
- First posted on arXiv 10 Dec 2024.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [DeiT III: Revenge of the ViT](https://mlsanity.com/paper/2204.07118/) (2022)
- [SpaceNet: A Remote Sensing Dataset and Challenge Series](https://mlsanity.com/paper/1807.01232/) (2018)
- [A Unified Sequence Interface for Vision Tasks](https://mlsanity.com/paper/2206.07669/) (2022)
- [Deep Snake for Real-Time Instance Segmentation](https://mlsanity.com/paper/2001.01629/) (2020)
- [Fast Interactive Object Annotation with Curve-GCN](https://mlsanity.com/paper/1903.06874/) (2019)
- [PolyTransform: Deep Polygon Transformer for Instance Segmentation](https://mlsanity.com/paper/1912.02801/) (2019)
- [Topological Map Extraction from Overhead Images](https://mlsanity.com/paper/1812.01497/) (2018)
- [PolyWorld: Polygonal Building Extraction with Graph Neural Networks in Satellite Images](https://mlsanity.com/paper/2111.15491/) (2021)
- [Learning deep structured active contours end-to-end](https://mlsanity.com/paper/1803.06329/) (2018)
- [DARNet: Deep Active Ray Network for Building Segmentation](https://mlsanity.com/paper/1905.05889/) (2019)
- [E2EC: An End-to-End Contour-based Method for High-Quality High-Speed Instance Segmentation](https://mlsanity.com/paper/2203.04074/) (2022)
- [RNGDet++: Road Network Graph Detection by Transformer with Instance Segmentation and Multi-scale Features Enhancement](https://mlsanity.com/paper/2209.10150/) (2022)
- [Machine-learned Regularization and Polygonization of Building Segmentation Masks](https://mlsanity.com/paper/2007.12587/) (2020)
- [Topo-boundary: A Benchmark Dataset on Topological Road-boundary Detection Using Aerial Images for Autonomous Driving](https://mlsanity.com/paper/2103.17119/) (2021)
- [BuildMapper: A Fully Learnable Framework for Vectorized Building Contour Extraction](https://mlsanity.com/paper/2211.03373/) (2022)
- [Accurate Polygonal Mapping of Buildings in Satellite Imagery](https://mlsanity.com/paper/2208.00609/) (2022)
- [HiT: Building Mapping with Hierarchical Transformers](https://mlsanity.com/paper/2309.09643/) (2023)
- [Data Leakage Detection and De-duplication in Large Scale Geospatial Image Datasets](https://mlsanity.com/paper/2304.02296/) (2023)

## Built on by (most cited first, 8 of 8)

- [ACPV-Net: All-Class Polygonal Vectorization for Seamless Vector Map Generation from Aerial Imagery](https://mlsanity.com/paper/2603.16616/) (2026)
- [Spatial Representation Learning Beyond Pixels: Unifying Raster Data and Vector Semantics for Human-Centric Geospatial Foundation Models](https://mlsanity.com/paper/2606.02374/) (2026)
- [AerialFusionMapNet: Online HD Map Construction with Aerial-Onboard BEV Fusion](https://mlsanity.com/paper/2606.24784/) (2026)
- [PCFootprint: A Large-Scale Dataset and Benchmark for Vectorized Building Footprint Extraction from Aerial LiDAR Point Clouds](https://mlsanity.com/paper/2606.20455/) (2026)
- [Click2Poly: A VLM for vector mapping buildings and walls](https://mlsanity.com/paper/2608.11424/) (2026)
- [SiZeUp: Fast 3D Proxy from Aerial Images via Depth Ordinal Loss](https://mlsanity.com/paper/2608.22821/) (2026)
- [PolyTopoBench: A Benchmark for Complex Vector Polygon Generation from Remote Sensing Imagery](https://mlsanity.com/paper/2609.32856/) (2026)
- [LoDEOT: Low-Dimensional and Efficient Offset Tokens for Building Footprint Extraction from Off-Nadir Imagery](https://mlsanity.com/paper/2610.05899/) (2026)

## Abstract (arXiv)

Extraction of building footprint polygons from remotely sensed data is essential for several urban understanding tasks such as reconstruction, navigation, and mapping. Despite significant progress in the area, extracting accurate polygonal building footprints remains an open problem. In this paper, we introduce Pix2Poly, an attention-based end-to-end trainable and differentiable deep neural network capable of directly generating explicit high-quality building footprints in a ring graph format. Pix2Poly employs a generative encoder-decoder transformer to produce a sequence of graph vertex tokens whose connectivity information is learned by an optimal matching network. Compared to previous graph learning methods, ours is a truly end-to-end trainable approach that extracts high-quality building footprints and road networks without requiring complicated, computationally intensive raster loss functions and intricate training pipelines. Upon evaluating Pix2Poly on several complex and challenging datasets, we report that Pix2Poly outperforms state-of-the-art methods in several vector shape quality metrics while being an entirely explicit method. Our code is available at https://github.com/yeshwanth95/Pix2Poly.