# Learning towards Minimum Hyperspherical Energy

arXiv: https://arxiv.org/abs/1805.09298 · HTML: https://mlsanity.com/paper/1805.09298/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2018.

## Built on by (most cited first, 30 of 75)

- [Controlling Text-to-Image Diffusion by Orthogonal Finetuning](https://mlsanity.com/paper/2306.07280/) (2023)
- [Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization](https://mlsanity.com/paper/2311.06243/) (2023)
- [Vision Transformers with Patch Diversification](https://mlsanity.com/paper/2104.12753/) (2021)
- [SphereFace2: Binary Classification is All You Need for Deep Face Recognition](https://mlsanity.com/paper/2108.01513/) (2021)
- [Learning with Hyperspherical Uniformity](https://mlsanity.com/paper/2103.01649/) (2021)
- [The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy](https://mlsanity.com/paper/2203.06345/) (2022)
- [Contrastive Syn-to-Real Generalization](https://mlsanity.com/paper/2104.02290/) (2021)
- [Ultra-High-Resolution Detector Simulation with Intra-Event Aware GAN and Self-Supervised Relational Reasoning](https://mlsanity.com/paper/2303.08046/) (2023)
- [Regular Polytope Networks](https://mlsanity.com/paper/2103.15632/) (2021)
- [SphereFace Revived: Unifying Hyperspherical Face Recognition](https://mlsanity.com/paper/2109.05565/) (2021)
- [SFace: Sigmoid-Constrained Hypersphere Loss for Robust Face Recognition](https://mlsanity.com/paper/2205.12010/) (2022)
- [Maximum Class Separation as Inductive Bias in One Matrix](https://mlsanity.com/paper/2206.08704/) (2022)
- [Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation](https://mlsanity.com/paper/2404.04316/) (2024)
- [PoLAR: Polar-Decomposed Low-Rank Adapter Representation](https://mlsanity.com/paper/2506.03133/) (2025)
- [Target-aware Molecular Graph Generation](https://mlsanity.com/paper/2202.04829/) (2022)
- [Hyperspherical Consistency Regularization](https://mlsanity.com/paper/2206.00845/) (2022)
- [Generalizing and Decoupling Neural Collapse via Hyperspherical Uniformity Gap](https://mlsanity.com/paper/2303.06484/) (2023)
- [Continual Learning by Modeling Intra-Class Variation](https://mlsanity.com/paper/2210.05398/) (2022)
- [Combating Representation Learning Disparity with Geometric Harmonization](https://mlsanity.com/paper/2310.17622/) (2023)
- [Iterative Teaching by Label Synthesis](https://mlsanity.com/paper/2110.14432/) (2021)
- [UniTSFace: Unified Threshold Integrated Sample-to-Sample Loss for Face Recognition](https://mlsanity.com/paper/2311.02523/) (2023)
- [3-in-1: 2D Rotary Adaptation for Efficient Finetuning, Efficient Batching and Composability](https://mlsanity.com/paper/2409.00119/) (2024)
- [FedUV: Uniformity and Variance for Heterogeneous Federated Learning](https://mlsanity.com/paper/2402.18372/) (2024)
- [Reparameterized LLM Training via Orthogonal Equivalence Transformation](https://mlsanity.com/paper/2506.08001/) (2025)
- [Iterative Teaching by Data Hallucination](https://mlsanity.com/paper/2210.17467/) (2022)
- [Spectral Scaling Laws in Language Models: How Effectively Do Feed-Forward Networks Use Their Latent Space?](https://mlsanity.com/paper/2510.00537/) (2025)
- [Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses](https://mlsanity.com/paper/2405.18045/) (2024)
- [Orthogonal Finetuning Made Scalable](https://mlsanity.com/paper/2506.19847/) (2025)
- [Low-distortion and GPU-compatible Tree Embeddings in Hyperbolic Space](https://mlsanity.com/paper/2502.17130/) (2025)
- [Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation](https://mlsanity.com/paper/2605.12492/) (2026)

## Abstract (arXiv)

Neural networks are a powerful class of nonlinear functions that can be trained end-to-end on various applications. While the over-parametrization nature in many neural networks renders the ability to fit complex functions and the strong representation power to handle challenging tasks, it also leads to highly correlated neurons that can hurt the generalization ability and incur unnecessary computation cost. As a result, how to regularize the network to avoid undesired representation redundancy becomes an important issue. To this end, we draw inspiration from a well-known problem in physics -- Thomson problem, where one seeks to find a state that distributes N electrons on a unit sphere as evenly as possible with minimum potential energy. In light of this intuition, we reduce the redundancy regularization problem to generic energy minimization, and propose a minimum hyperspherical energy (MHE) objective as generic regularization for neural networks. We also propose a few novel variants of MHE, and provide some insights from a theoretical point of view. Finally, we apply neural networks with MHE regularization to several challenging tasks. Extensive experiments demonstrate the effectiveness of our intuition, by showing the superior performance with MHE regularization.