PolyTransform: Deep Polygon Transformer for Instance Segmentation
Not on the map. Why?
Abstract (arXiv)
In this paper, we propose PolyTransform, a novel instance segmentation algorithm that produces precise, geometry-preserving masks by combining the strengths of prevailing segmentation approaches and modern polygon-based methods. In particular, we first exploit a segmentation network to generate instance masks. We then convert the masks into a set of polygons that are then fed to a deforming network that transforms the polygons such that they better fit the object boundaries. Our experiments on the challenging Cityscapes dataset show that our PolyTransform significantly improves the performance of the backbone instance segmentation network and ranks 1st on the Cityscapes test-set leaderboard. We also show impressive gains in the interactive annotation setting. We release the code at https://github.com/uber-research/PolyTransform.
Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Dec 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 32; 2022: 22; 2023: 16; 2024: 14; 2025: 5; 2026: 3.
92 papers.
Sort:
By area:
- 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 311
- 2021SegDiff: Image Segmentation with Diffusion Probabilistic Modelscited by 279
- 2022VectorMapNet: End-to-end Vectorized HD Map Learningcited by 243
- 2021Boundary IoU: Improving Object-Centric Image Segmentation Evaluationcited by 186
- 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 186
- 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
- 2021MSeg: A Composite Dataset for Multi-domain Semantic Segmentationcited by 122
- 2021Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understandingcited by 97
- 2021TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Trackingcited by 93
- 2022MiniViT: Compressing Vision Transformers with Weight Multiplexingcited by 88
- 2022Few-Shot Class-Incremental Learning by Sampling Multi-Phase Taskscited by 87
- 2021Crossover Learning for Fast Online Video Instance Segmentationcited by 83
38 more
- 2021Mask Transfiner for High-Quality Instance Segmentationcited by 64
- 2022Waymo Open Dataset: Panoramic Video Panoptic Segmentationcited by 57
- 2021Part-aware Panoptic Segmentationcited by 56
- 2024MGMap: Mask-Guided Learning for Online Vectorized HD Map Constructioncited by 52
- 2023C-SFDA: A Curriculum Learning Aided Self-Training Framework for Efficient Source Free Domain Adaptationcited by 49
- 2022Connecting the Dots: Floorplan Reconstruction Using Two-Level Queriescited by 48
- 2021RefineMask: Towards High-Quality Instance Segmentation with Fine-Grained Featurescited by 47
- 2021Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentationcited by 43
- 2021PolyWorld: Polygonal Building Extraction with Graph Neural Networks in Satellite Imagescited by 42
- 2022RBGNet: Ray-based Grouping for 3D Object Detectioncited by 40
- 2022Panoptic-PartFormer: Learning a Unified Model for Panoptic Part Segmentationcited by 38
- 2021PlaneTR: Structure-Guided Transformers for 3D Plane Recoverycited by 33
- 2024TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learningcited by 29
- 2021Transformer Meets Convolution: A Bilateral Awareness Network for Semantic Segmentation of Very Fine Resolution Urban Scene Imagescited by 28
- 2023SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete Diffusion Processcited by 28
- 2022E2EC: An End-to-End Contour-based Method for High-Quality High-Speed Instance Segmentationcited by 27
- 2022MGNet: Monocular Geometric Scene Understanding for Autonomous Drivingcited by 27
- 2022Towards General-Purpose Representation Learning of Polygonal Geometriescited by 21
- 2021Weakly Supervised Instance Segmentation for Videos with Temporal Mask Consistencycited by 19
- 2023PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentationcited by 19
- 2021Unknown Object Segmentation from Stereo Imagescited by 16
- 2023ScalableMap: Scalable Map Learning for Online Long-Range Vectorized HD Map Constructioncited by 16
- 2023Transformer as Linear Expansion of Learngenecited by 13
- 2021Adversarial Shape Learning for Building Extraction in VHR Remote Sensing Imagescited by 12
- 2021A State-of-the-art Survey of Object Detection Techniques in Microorganism Image Analysis: From Classical Methods to Deep Learning Approachescited by 12
- 2021Infrared Small-Dim Target Detection with Transformer under Complex Backgroundscited by 12
- 2022SharpContour: A Contour-based Boundary Refinement Approach for Efficient and Accurate Instance Segmentationcited by 12
- 2023BoxSnake: Polygonal Instance Segmentation with Box Supervisioncited by 12
- 2022Multilevel Hierarchical Network with Multiscale Sampling for Video Question Answeringcited by 11
- 2022BuildMapper: A Fully Learnable Framework for Vectorized Building Contour Extractioncited by 11
- 2023EWT: Efficient Wavelet-Transformer for Single Image Denoisingcited by 11
- 2022RF-Next: Efficient Receptive Field Search for Convolutional Neural Networkscited by 10
- 2022Visual Recognition by Requestcited by 10
- 2023Recurrent Generic Contour-based Instance Segmentation with Progressive Learningcited by 9
- 2023Milestones in Autonomous Driving and Intelligent Vehicles Part II: Perception and Planningcited by 9
- 2021VideoClick: Video Object Segmentation with a Single Clickcited by 8
- 2024PolyRoom: Room-aware Transformer for Floorplan Reconstructioncited by 8
- 2021INSTA-YOLO: Real-Time Instance Segmentationcited by 7
50 most cited of 92 All 92 (JSON)
- 2026PolyTopoBench: A Benchmark for Complex Vector Polygon Generation from Remote Sensing Imagery26 Sep 2026
- 2026Topology-Preserving Polygon Augmentation for Segmentation in Structured Visual Domains16 Mar 2026
- 2026Tokenizing Semantic Segmentation with Run Length Encoding25 Feb 2026
- 2025HoliTracer: Holistic Vectorization of Geographic Objects from Large-Size Remote Sensing Imagery22 Jul 2025
- 2025Unified Medical Image Segmentation with State Space Modeling Snake17 Jul 2025
- 2025MARL-MambaContour: Unleashing Multi-Agent Deep Reinforcement Learning for Active Contour Optimization in Medical Image Segmentation23 Jun 2025
- 2025Uncertainty-Masked Bernoulli Diffusion for Camouflaged Object Detection Refinement12 Jun 2025
- 2025COIN: Confidence Score-Guided Distillation for Annotation-Free Cell Segmentation14 Mar 2025
- 2024Pix2Poly: A Sequence Prediction Method for End-to-end Polygonal Building Footprint Extraction from Remote Sensing Imagery10 Dec 2024
- 2024TreeFormer: Single-view Plant Skeleton Estimation via Tree-constrained Graph Generation25 Nov 2024
- 2024VisDiff: SDF-Guided Polygon Generation for Visibility Reconstruction and Recognition7 Oct 2024
- 2024RoIPoly: Vectorized Building Outline Extraction Using Vertex and Logit Embeddings20 Jul 2024
38 more
- 2024PolyRoom: Room-aware Transformer for Floorplan Reconstruction15 Jul 2024
- 2024TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning21 Jun 2024
- 2024P2PFormer: A Primitive-to-polygon Method for Regular Building Contour Extraction from Remote Sensing Images5 Jun 2024
- 2024DenseSeg: Joint Learning for Semantic Segmentation and Landmark Detection Using Dense Image-to-Shape Representation30 May 2024
- 2024High-order Neighborhoods Know More: HyperGraph Learning Meets Source-free Unsupervised Domain Adaptation11 May 2024
- 2024AdaContour: Adaptive Contour Descriptor with Hierarchical Representation12 Apr 2024
- 2024MGMap: Mask-Guided Learning for Online Vectorized HD Map Construction1 Apr 2024
- 2024CenterDisks: Real-time instance segmentation with disk covering5 Mar 2024
- 2024Source-Free Unsupervised Domain Adaptation with Hypothesis Consolidation of Prediction Rationale2 Feb 2024
- 2024Combining Image- and Geometric-based Deep Learning for Shape Regression: A Comparison to Pixel-level Methods for Segmentation in Chest X-Ray15 Jan 2024
- 2023SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete Diffusion Process19 Dec 2023
- 2023Towards Effective Multi-Moving-Camera Tracking: A New Dataset and Lightweight Link Model18 Dec 2023
- 2023MatchDet: A Collaborative Framework for Image Matching and Object Detection18 Dec 2023
- 2023Transformer as Linear Expansion of Learngene9 Dec 2023
- 2023ScalableMap: Scalable Map Learning for Online Long-Range Vectorized HD Map Construction20 Oct 2023
- 2023Transferable Attack for Semantic Segmentation31 Jul 2023
- 2023Milestones in Autonomous Driving and Intelligent Vehicles Part II: Perception and Planning3 Jun 2023
- 2023Real-time instance segmentation with polygons using an Intersection-over-Union loss9 May 2023
- 2023Clustered-patch Element Connection for Few-shot Learning20 Apr 2023
- 2023EWT: Efficient Wavelet-Transformer for Single Image Denoising13 Apr 2023
- 2023C-SFDA: A Curriculum Learning Aided Self-Training Framework for Efficient Source Free Domain Adaptation30 Mar 2023
- 2023BoxSnake: Polygonal Instance Segmentation with Box Supervision21 Mar 2023
- 2023BiSVP: Building Footprint Extraction via Bidirectional Serialized Vertex Prediction1 Mar 2023
- 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generation14 Feb 2023
- 2023Recurrent Generic Contour-based Instance Segmentation with Progressive Learning21 Jan 2023
- 2023PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation3 Jan 2023
- 2022Framework for 2D Ad placements in LinearTV5 Dec 2022
- 2022Connecting the Dots: Floorplan Reconstruction Using Two-Level Queries28 Nov 2022
- 2022BuildMapper: A Fully Learnable Framework for Vectorized Building Contour Extraction7 Nov 2022
- 2022tSF: Transformer-based Semantic Filter for Few-Shot Learning2 Nov 2022
- 2022Towards General-Purpose Representation Learning of Polygonal Geometries29 Sep 2022
- 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction30 Aug 2022
- 2022Visual Recognition by Request28 Jul 2022
- 2022Primitive Graph Learning for Unified Vector Mapping28 Jun 2022
- 2022MGNet: Monocular Geometric Scene Understanding for Autonomous Driving27 Jun 2022
- 2022Deep Active Latent Surfaces for Medical Geometries21 Jun 2022
- 2022VectorMapNet: End-to-end Vectorized HD Map Learning17 Jun 2022
- 2022Waymo Open Dataset: Panoramic Video Panoptic Segmentation15 Jun 2022
50 newest of 92 All 92 (JSON)
- 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 311
- 2022VectorMapNet: End-to-end Vectorized HD Map Learningcited by 243
- 2021Boundary IoU: Improving Object-Centric Image Segmentation Evaluationcited by 186
- 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 186
- 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
- 2021MSeg: A Composite Dataset for Multi-domain Semantic Segmentationcited by 122
- 2021Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understandingcited by 97
- 2021TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Trackingcited by 93
- 2022MiniViT: Compressing Vision Transformers with Weight Multiplexingcited by 88
- 2022Few-Shot Class-Incremental Learning by Sampling Multi-Phase Taskscited by 87
- 2021Crossover Learning for Fast Online Video Instance Segmentationcited by 83
- 2021Mask Transfiner for High-Quality Instance Segmentationcited by 64
12 most cited of 89 All 89 (JSON)
- 2026PolyTopoBench: A Benchmark for Complex Vector Polygon Generation from Remote Sensing Imagery26 Sep 2026
- 2026Topology-Preserving Polygon Augmentation for Segmentation in Structured Visual Domains16 Mar 2026
- 2026Tokenizing Semantic Segmentation with Run Length Encoding25 Feb 2026
- 2025HoliTracer: Holistic Vectorization of Geographic Objects from Large-Size Remote Sensing Imagery22 Jul 2025
- 2025MARL-MambaContour: Unleashing Multi-Agent Deep Reinforcement Learning for Active Contour Optimization in Medical Image Segmentation23 Jun 2025
- 2025Uncertainty-Masked Bernoulli Diffusion for Camouflaged Object Detection Refinement12 Jun 2025
- 2025COIN: Confidence Score-Guided Distillation for Annotation-Free Cell Segmentation14 Mar 2025
- 2024Pix2Poly: A Sequence Prediction Method for End-to-end Polygonal Building Footprint Extraction from Remote Sensing Imagery10 Dec 2024
- 2024TreeFormer: Single-view Plant Skeleton Estimation via Tree-constrained Graph Generation25 Nov 2024
- 2024VisDiff: SDF-Guided Polygon Generation for Visibility Reconstruction and Recognition7 Oct 2024
- 2024RoIPoly: Vectorized Building Outline Extraction Using Vertex and Logit Embeddings20 Jul 2024
- 2024PolyRoom: Room-aware Transformer for Floorplan Reconstruction15 Jul 2024
12 newest of 89 All 89 (JSON)