# BoxSnake: Polygonal Instance Segmentation with Box Supervision

arXiv: https://arxiv.org/abs/2303.11630 · HTML: https://mlsanity.com/paper/2303.11630/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 89 references that are in the map.
- First posted on arXiv 21 Mar 2023.

## Builds on (most cited first, 30 of 58)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression](https://mlsanity.com/paper/1911.08287/) (2019)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [Cascade R-CNN: High Quality Object Detection and Instance Segmentation](https://mlsanity.com/paper/1906.09756/) (2019)
- [Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1803.10464/) (2018)
- [Conditional Convolutions for Instance Segmentation](https://mlsanity.com/paper/2003.05664/) (2020)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1503.01640/) (2015)
- [K-Net: Towards Unified Image Segmentation](https://mlsanity.com/paper/2106.14855/) (2021)
- [PolarMask: Single Shot Instance Segmentation with Polar Representation](https://mlsanity.com/paper/1909.13226/) (2019)
- [Instances as Queries](https://mlsanity.com/paper/2105.01928/) (2021)
- [A Unified Sequence Interface for Vision Tasks](https://mlsanity.com/paper/2206.07669/) (2022)
- [BoxInst: High-Performance Instance Segmentation with Box Annotations](https://mlsanity.com/paper/2012.02310/) (2020)
- [On Regularized Losses for Weakly-supervised CNN Segmentation](https://mlsanity.com/paper/1803.09569/) (2018)
- [Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++](https://mlsanity.com/paper/1803.09693/) (2018)
- [Normalized Cut Loss for Weakly-supervised CNN Segmentation](https://mlsanity.com/paper/1804.01346/) (2018)

## Built on by (most cited first, 12 of 12)

- [GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction](https://mlsanity.com/paper/2305.18752/) (2023)
- [Weakly-Supervised Concealed Object Segmentation with SAM-based Pseudo Labeling and Multi-scale Feature Grouping](https://mlsanity.com/paper/2305.11003/) (2023)
- [Meta-Adapter: An Online Few-shot Learner for Vision-Language Model](https://mlsanity.com/paper/2311.03774/) (2023)
- [GrootVL: Tree Topology is All You Need in State Space Model](https://mlsanity.com/paper/2406.02395/) (2024)
- [UniHead: Unifying Multi-Perception for Detection Heads](https://mlsanity.com/paper/2309.13242/) (2023)
- [Label-efficient Segmentation via Affinity Propagation](https://mlsanity.com/paper/2310.10533/) (2023)
- [Detecting Wildfires on UAVs with Real-time Segmentation Trained by Larger Teacher Models](https://mlsanity.com/paper/2408.10843/) (2024)
- [Video Object Segmentation with Dynamic Query Modulation](https://mlsanity.com/paper/2403.11529/) (2024)
- [BoxCell: Leveraging SAM for Cell Segmentation with Box Supervision](https://mlsanity.com/paper/2311.17960/) (2023)
- [Advancing SEM Based Nano-Scale Defect Analysis in Semiconductor Manufacturing for Advanced IC Nodes](https://mlsanity.com/paper/2409.04310/) (2024)
- [SAPNet++: Evolving Point-Prompted Instance Segmentation with Semantic and Spatial Awareness](https://mlsanity.com/paper/2602.21762/) (2026)
- [Noise-Robust Box-Supervised Infrared Small Target Detection via Physics-Inspired Soft Label Optimization](https://mlsanity.com/paper/2607.17148/) (2026)

## Abstract (arXiv)

Box-supervised instance segmentation has gained much attention as it requires only simple box annotations instead of costly mask or polygon annotations. However, existing box-supervised instance segmentation models mainly focus on mask-based frameworks. We propose a new end-to-end training technique, termed BoxSnake, to achieve effective polygonal instance segmentation using only box annotations for the first time. Our method consists of two loss functions: (1) a point-based unary loss that constrains the bounding box of predicted polygons to achieve coarse-grained segmentation; and (2) a distance-aware pairwise loss that encourages the predicted polygons to fit the object boundaries. Compared with the mask-based weakly-supervised methods, BoxSnake further reduces the performance gap between the predicted segmentation and the bounding box, and shows significant superiority on the Cityscapes dataset. The code has been available publicly.