# DeepSkeleton: Learning Multi-task Scale-associated Deep Side Outputs for Object Skeleton Extraction in Natural Images

arXiv: https://arxiv.org/abs/1609.03659 · HTML: https://mlsanity.com/paper/1609.03659/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Sep 2016.

## Built on by (most cited first, 13 of 13)

- [A skeletonization algorithm for gradient-based optimization](https://mlsanity.com/paper/2309.02527/) (2023)
- [WAS: Dataset and Methods for Artistic Text Segmentation](https://mlsanity.com/paper/2408.00106/) (2024)
- [Structure-Preserving Instance Segmentation via Skeleton-Aware Distance Transform](https://mlsanity.com/paper/2310.05262/) (2023)
- [Emergence of Double-slit Interference by Representing Visual Space in Artificial Neural Networks](https://mlsanity.com/paper/2205.10081/) (2022)
- [Context Attention Network for Skeleton Extraction](https://mlsanity.com/paper/2205.12066/) (2022)
- [Skeletal Point Representations with Geometric Deep Learning](https://mlsanity.com/paper/2303.02123/) (2023)
- [Skeleton Ground Truth Extraction: Methodology, Annotation Tool and Benchmarks](https://mlsanity.com/paper/2310.06437/) (2023)
- [Mode-locking Theory for Long-Range Interaction in Artificial Neural Networks](https://mlsanity.com/paper/2303.05695/) (2023)
- [Fitting Skeletal Models via Graph-based Learning](https://mlsanity.com/paper/2409.05311/) (2024)
- [Topology-Aware Skeleton Detection via Lighthouse-Guided Structured Inference](https://mlsanity.com/paper/2604.20123/) (2026)
- [Continuous-tone Simple Points: An ℓ0-Norm of Cyclic Gradient for Topology-Preserving Data-Driven Image Segmentation](https://mlsanity.com/paper/2604.28159/) (2026)
- [Multimodal Plant Root Phenotyping with Integration of 3D Skeleton Extraction and Language Analysis](https://mlsanity.com/paper/2608.03109/) (2026)
- [Depth-Dominant Skeleton Detection for Natural Scenes](https://mlsanity.com/paper/2608.16367/) (2026)

## Abstract (arXiv)

Object skeletons are useful for object representation and object detection. They are complementary to the object contour, and provide extra information, such as how object scale (thickness) varies among object parts. But object skeleton extraction from natural images is very challenging, because it requires the extractor to be able to capture both local and non-local image context in order to determine the scale of each skeleton pixel. In this paper, we present a novel fully convolutional network with multiple scale-associated side outputs to address this problem. By observing the relationship between the receptive field sizes of the different layers in the network and the skeleton scales they can capture, we introduce two scale-associated side outputs to each stage of the network. The network is trained by multi-task learning, where one task is skeleton localization to classify whether a pixel is a skeleton pixel or not, and the other is skeleton scale prediction to regress the scale of each skeleton pixel. Supervision is imposed at different stages by guiding the scale-associated side outputs toward the groundtruth skeletons at the appropriate scales. The responses of the multiple scale-associated side outputs are then fused in a scale-specific way to detect skeleton pixels using multiple scales effectively. Our method achieves promising results on two skeleton extraction datasets, and significantly outperforms other competitors. Additionally, the usefulness of the obtained skeletons and scales (thickness) are verified on two object detection applications: Foreground object segmentation and object proposal detection.