# PolyWorld: Polygonal Building Extraction with Graph Neural Networks in Satellite Images

arXiv: https://arxiv.org/abs/2111.15491 · HTML: https://mlsanity.com/paper/2111.15491/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 43 references that are in the map.
- First posted on arXiv 30 Nov 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [On Regularized Losses for Weakly-supervised CNN Segmentation](https://mlsanity.com/paper/1803.09569/) (2018)
- [Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++](https://mlsanity.com/paper/1803.09693/) (2018)
- [Normalized Cut Loss for Weakly-supervised CNN Segmentation](https://mlsanity.com/paper/1804.01346/) (2018)
- [Fast Interactive Object Annotation with Curve-GCN](https://mlsanity.com/paper/1903.06874/) (2019)
- [PolyTransform: Deep Polygon Transformer for Instance Segmentation](https://mlsanity.com/paper/1912.02801/) (2019)
- [Annotating Object Instances with a Polygon-RNN](https://mlsanity.com/paper/1704.05548/) (2017)
- [TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image Segmentation](https://mlsanity.com/paper/1801.05746/) (2018)
- [Topological Map Extraction from Overhead Images](https://mlsanity.com/paper/1812.01497/) (2018)
- [Learning deep structured active contours end-to-end](https://mlsanity.com/paper/1803.06329/) (2018)
- [DARNet: Deep Active Ray Network for Building Segmentation](https://mlsanity.com/paper/1905.05889/) (2019)
- [MonteFloor: Extending MCTS for Reconstructing Accurate Large-Scale Floor Plans](https://mlsanity.com/paper/2103.11161/) (2021)
- [TernausNetV2: Fully Convolutional Network for Instance Segmentation](https://mlsanity.com/paper/1806.00844/) (2018)
- [Machine-learned Regularization and Polygonization of Building Segmentation Masks](https://mlsanity.com/paper/2007.12587/) (2020)
- [Regularization of Building Boundaries in Satellite Images using Adversarial and Regularized Losses](https://mlsanity.com/paper/2007.11840/) (2020)
- [Quantization in Relative Gradient Angle Domain For Building Polygon Estimation](https://mlsanity.com/paper/2007.05617/) (2020)

## Built on by (most cited first, 30 of 43)

- [VectorMapNet: End-to-end Vectorized HD Map Learning](https://mlsanity.com/paper/2206.08920/) (2022)
- [SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery](https://mlsanity.com/paper/2312.10115/) (2023)
- [SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understanding](https://mlsanity.com/paper/2211.15660/) (2022)
- [InstaGraM: Instance-level Graph Modeling for Vectorized HD Map Learning](https://mlsanity.com/paper/2301.04470/) (2023)
- [ScalableMap: Scalable Map Learning for Online Long-Range Vectorized HD Map Construction](https://mlsanity.com/paper/2310.13378/) (2023)
- [GlobalMapper: Arbitrary-Shaped Urban Layout Generation](https://mlsanity.com/paper/2307.09693/) (2023)
- [3D Building Reconstruction from Monocular Remote Sensing Images with Multi-level Supervisions](https://mlsanity.com/paper/2404.04823/) (2024)
- [PolygonGNN: Representation Learning for Polygonal Geometries with Heterogeneous Visibility Graph](https://mlsanity.com/paper/2407.00742/) (2024)
- [Pix2Poly: A Sequence Prediction Method for End-to-end Polygonal Building Footprint Extraction from Remote Sensing Imagery](https://mlsanity.com/paper/2412.07899/) (2024)
- [Accurate Polygonal Mapping of Buildings in Satellite Imagery](https://mlsanity.com/paper/2208.00609/) (2022)
- [HiT: Building Mapping with Hierarchical Transformers](https://mlsanity.com/paper/2309.09643/) (2023)
- [PolyR-CNN: R-CNN for end-to-end polygonal building outline extraction](https://mlsanity.com/paper/2407.14912/) (2024)
- [RoIPoly: Vectorized Building Outline Extraction Using Vertex and Logit Embeddings](https://mlsanity.com/paper/2407.14920/) (2024)
- [Patched Line Segment Learning for Vector Road Mapping](https://mlsanity.com/paper/2309.02923/) (2023)
- [P2PFormer: A Primitive-to-polygon Method for Regular Building Contour Extraction from Remote Sensing Images](https://mlsanity.com/paper/2406.02930/) (2024)
- [Data Leakage Detection and De-duplication in Large Scale Geospatial Image Datasets](https://mlsanity.com/paper/2304.02296/) (2023)
- [Global Collinearity-aware Polygonizer for Polygonal Building Mapping in Remote Sensing](https://mlsanity.com/paper/2505.01385/) (2025)
- [HoliTracer: Holistic Vectorization of Geographic Objects from Large-Size Remote Sensing Imagery](https://mlsanity.com/paper/2507.16251/) (2025)
- [PolyBuilding: Polygon Transformer for End-to-End Building Extraction](https://mlsanity.com/paper/2211.01589/) (2022)
- [Beyond Grid Data: Exploring Graph Neural Networks for Earth Observation](https://mlsanity.com/paper/2411.03223/) (2024)
- [LDPoly: Latent Diffusion for Polygonal Road Outline Extraction in Large-Scale Topographic Mapping](https://mlsanity.com/paper/2504.20645/) (2025)
- [ACPV-Net: All-Class Polygonal Vectorization for Seamless Vector Map Generation from Aerial Imagery](https://mlsanity.com/paper/2603.16616/) (2026)
- [BiSVP: Building Footprint Extraction via Bidirectional Serialized Vertex Prediction](https://mlsanity.com/paper/2303.00300/) (2023)
- [3D detection of roof sections from a single satellite image and application to LOD2-building reconstruction](https://mlsanity.com/paper/2307.05409/) (2023)
- [GeoFormer: A Multi-Polygon Segmentation Transformer](https://mlsanity.com/paper/2411.16616/) (2024)
- [SAR Object Detection with Self-Supervised Pretraining and Curriculum-Aware Sampling](https://mlsanity.com/paper/2504.13310/) (2025)
- [The P3 Dataset: Pixels, Points and Polygons for Multimodal Building Vectorization](https://mlsanity.com/paper/2505.15379/) (2025)
- [Bridging the Gap Between Sparsity and Redundancy: A Dual-Decoding Framework with Global Context for Map Inference](https://mlsanity.com/paper/2509.11731/) (2025)
- [UniVector: Unified Vector Extraction via Instance-Geometry Interaction](https://mlsanity.com/paper/2510.13234/) (2025)
- [Primitive Graph Learning for Unified Vector Mapping](https://mlsanity.com/paper/2206.13963/) (2022)

## Abstract (arXiv)

While most state-of-the-art instance segmentation methods produce binary segmentation masks, geographic and cartographic applications typically require precise vector polygons of extracted objects instead of rasterized output. This paper introduces PolyWorld, a neural network that directly extracts building vertices from an image and connects them correctly to create precise polygons. The model predicts the connection strength between each pair of vertices using a graph neural network and estimates the assignments by solving a differentiable optimal transport problem. Moreover, the vertex positions are optimized by minimizing a combined segmentation and polygonal angle difference loss. PolyWorld significantly outperforms the state of the art in building polygonization and achieves not only notable quantitative results, but also produces visually pleasing building polygons. Code and trained weights are publicly available at https://github.com/zorzi-s/PolyWorldPretrainedNetwork.