# Data-Efficient Learning via Minimizing Hyperspherical Energy

arXiv: https://arxiv.org/abs/2206.15204 · HTML: https://mlsanity.com/paper/2206.15204/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 86 references that are in the map.
- First posted on arXiv 30 Jun 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data](https://mlsanity.com/paper/2002.11297/) (2020)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [Contextual Diversity for Active Learning](https://mlsanity.com/paper/2008.05723/) (2020)
- [Learning From Noisy Large-Scale Datasets With Minimal Supervision](https://mlsanity.com/paper/1701.01619/) (2017)
- [Hyperspherical Prototype Networks](https://mlsanity.com/paper/1901.10514/) (2019)
- [Learning towards Minimum Hyperspherical Energy](https://mlsanity.com/paper/1805.09298/) (2018)
- [Deep Hyperspherical Learning](https://mlsanity.com/paper/1711.03189/) (2017)
- [Differentiating through the Fr'echet Mean](https://mlsanity.com/paper/2003.00335/) (2020)
- [Decoupled Networks](https://mlsanity.com/paper/1804.08071/) (2018)
- [Orthogonal Over-Parameterized Training](https://mlsanity.com/paper/2004.04690/) (2020)
- [Learning with Hyperspherical Uniformity](https://mlsanity.com/paper/2103.01649/) (2021)
- [Large Margin Softmax Loss for Speaker Verification](https://mlsanity.com/paper/1904.03479/) (2019)
- [Angular Visual Hardness](https://mlsanity.com/paper/1912.02279/) (2019)
- [Regularizing Neural Networks via Minimizing Hyperspherical Energy](https://mlsanity.com/paper/1906.04892/) (2019)
- [Beyond Disagreement-based Agnostic Active Learning](https://mlsanity.com/paper/1407.2657/) (2014)
- [Exploring Representativeness and Informativeness for Active Learning](https://mlsanity.com/paper/1904.06685/) (2019)
- [Generative Modeling for Small-Data Object Detection](https://mlsanity.com/paper/1910.07169/) (2019)
- [Efficient and Parsimonious Agnostic Active Learning](https://mlsanity.com/paper/1506.08669/) (2015)
- [Neural Similarity Learning](https://mlsanity.com/paper/1910.13003/) (2019)
- [Finding the Homology of Decision Boundaries with Active Learning](https://mlsanity.com/paper/2011.09645/) (2020)
- [Adaptive Region-Based Active Learning](https://mlsanity.com/paper/2002.07348/) (2020)

## Built on by (most cited first, 4 of 4)

- [Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigation](https://mlsanity.com/paper/2510.08962/) (2025)
- [UtilGen: Utility-Centric Generative Data Augmentation with Dual-Level Task Adaptation](https://mlsanity.com/paper/2510.24262/) (2025)
- [Do All Individual Layers Help? An Empirical Study of Task-Interfering Layers in Vision-Language Models](https://mlsanity.com/paper/2602.01167/) (2026)
- [A Survey of Learning on Small Data: Generalization, Optimization, and Challenge](https://mlsanity.com/paper/2207.14443/) (2022)

## Abstract (arXiv)

Deep learning on large-scale data is dominant nowadays. The unprecedented scale of data has been arguably one of the most important driving forces for the success of deep learning. However, there still exist scenarios where collecting data or labels could be extremely expensive, e.g., medical imaging and robotics. To fill up this gap, this paper considers the problem of data-efficient learning from scratch using a small amount of representative data. First, we characterize this problem by active learning on homeomorphic tubes of spherical manifolds. This naturally generates feasible hypothesis class. With homologous topological properties, we identify an important connection -- finding tube manifolds is equivalent to minimizing hyperspherical energy (MHE) in physical geometry. Inspired by this connection, we propose a MHE-based active learning (MHEAL) algorithm, and provide comprehensive theoretical guarantees for MHEAL, covering convergence and generalization analysis. Finally, we demonstrate the empirical performance of MHEAL in a wide range of applications on data-efficient learning, including deep clustering, distribution matching, version space sampling and deep active learning.