# HERS Superpixels: Deep Affinity Learning for Hierarchical Entropy Rate Segmentation

arXiv: https://arxiv.org/abs/2106.03755 · HTML: https://mlsanity.com/paper/2106.03755/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 50 references that are in the map.
- First posted on arXiv 7 Jun 2021.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [SelFlow: Self-Supervised Learning of Optical Flow](https://mlsanity.com/paper/1904.09117/) (2019)
- [Superpixel Sampling Networks](https://mlsanity.com/paper/1807.10174/) (2018)
- [SEEDS: Superpixels Extracted via Energy-Driven Sampling](https://mlsanity.com/paper/1309.3848/) (2013)
- [Superpixel Segmentation with Fully Convolutional Networks](https://mlsanity.com/paper/2003.12929/) (2020)
- [Superpixel Convolutional Networks using Bilateral Inceptions](https://mlsanity.com/paper/1511.06739/) (2015)
- [Superpixel Hierarchy](https://mlsanity.com/paper/1605.06325/) (2016)
- [Superpixel Contracted Graph-Based Learning for Hyperspectral Image Classification](https://mlsanity.com/paper/1903.06548/) (2019)
- [Peekaboo - Where are the Objects? Structure Adjusting Superpixels](https://mlsanity.com/paper/1802.02796/) (2018)
- [Dynamic Spectral Residual Superpixels](https://mlsanity.com/paper/1910.04794/) (2019)

## Built on by (most cited first, 8 of 8)

- [A comprehensive review and new taxonomy on superpixel segmentation](https://mlsanity.com/paper/2409.19179/) (2024)
- [Efficient Multiscale Object-based Superpixel Framework](https://mlsanity.com/paper/2204.03533/) (2022)
- [Superpixel Anything: A general object-based framework for accurate yet regular superpixel segmentation](https://mlsanity.com/paper/2509.12791/) (2025)
- [Superpixel Segmentation: A Long-Lasting Ill-Posed Problem](https://mlsanity.com/paper/2411.06478/) (2024)
- [Learning Semantic Segmentation with Query Points Supervision on Aerial Images](https://mlsanity.com/paper/2309.05490/) (2023)
- [Superpixel Integrated Grids for Fast Image Segmentation](https://mlsanity.com/paper/2510.06487/) (2025)
- [Computer Vision with a Superpixelation Camera](https://mlsanity.com/paper/2603.26900/) (2026)
- [H-SPAM: Hierarchical Superpixel Anything Model](https://mlsanity.com/paper/2604.11218/) (2026)

## Abstract (arXiv)

Superpixels serve as a powerful preprocessing tool in numerous computer vision tasks. By using superpixel representation, the number of image primitives can be largely reduced by orders of magnitudes. With the rise of deep learning in recent years, a few works have attempted to feed deeply learned features / graphs into existing classical superpixel techniques. However, none of them are able to produce superpixels in near real-time, which is crucial to the applicability of superpixels in practice. In this work, we propose a two-stage graph-based framework for superpixel segmentation. In the first stage, we introduce an efficient Deep Affinity Learning (DAL) network that learns pairwise pixel affinities by aggregating multi-scale information. In the second stage, we propose a highly efficient superpixel method called Hierarchical Entropy Rate Segmentation (HERS). Using the learned affinities from the first stage, HERS builds a hierarchical tree structure that can produce any number of highly adaptive superpixels instantaneously. We demonstrate, through visual and numerical experiments, the effectiveness and efficiency of our method compared to various state-of-the-art superpixel methods.