mlsanityData to 7 Oct 2026

Learning with Hyperspherical Uniformity

arXiv 2103.01649 ↗First posted 2 Mar 2021cs.LG cs.CVcited by 35

Not on the map. Why?

Abstract (arXiv)

Due to the over-parameterization nature, neural networks are a powerful tool for nonlinear function approximation. In order to achieve good generalization on unseen data, a suitable inductive bias is of great importance for neural networks. One of the most straightforward ways is to regularize the neural network with some additional objectives. L2 regularization serves as a standard regularization for neural networks. Despite its popularity, it essentially regularizes one dimension of the individual neuron, which is not strong enough to control the capacity of highly over-parameterized neural networks. Motivated by this, hyperspherical uniformity is proposed as a novel family of relational regularizations that impact the interaction among neurons. We consider several geometrically distinct ways to achieve hyperspherical uniformity. The effectiveness of hyperspherical uniformity is justified by theoretical insights and empirical evaluations.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 38 of its 91 references that are in the map. First posted on arXiv 2 Mar 2021.

Builds on

38 of 91 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  7. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  8. 2016Wide Residual Networkscited by 3,057
  9. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  10. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  11. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  12. 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 968
26 more
  1. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 733
  2. 2016Learning to learn by gradient descent by gradient descentcited by 616
  3. 2019Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processingcited by 411
  4. 2016Large-Margin Softmax Loss for Convolutional Neural Networkscited by 296
  5. 2015Unitary Evolution Recurrent Neural Networkscited by 291
  6. 2018Additive Margin Softmax for Face Verificationcited by 275
  7. 2018Hyperspherical Variational Auto-Encoderscited by 188
  8. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 178
  9. 2016Neural Photo Editing with Introspective Adversarial Networkscited by 109
  10. 2015Convolutional neural networks with low-rank regularizationcited by 102
  11. 2015Reducing Overfitting in Deep Networks by Decorrelating Representationscited by 94
  12. 2017Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networkscited by 92
  13. 2016Full-Capacity Unitary Recurrent Neural Networkscited by 90
  14. 2016Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Unitscited by 86
  15. 2018Learning towards Minimum Hyperspherical Energycited by 75
  16. 2019Orthogonal Deep Neural Networkscited by 72
  17. 2016Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNscited by 62
  18. 2017Deep Hyperspherical Learningcited by 55
  19. 2018Decoupled Networkscited by 45
  20. 2020Orthogonal Over-Parameterized Trainingcited by 44
  21. 2016Regularizing CNNs with Locally Constrained Decorrelationscited by 41
  22. 2020Deep Isometric Learning for Visual Recognitioncited by 38
  23. 2016Diverse Neural Network Learns True Target Functionscited by 33
  24. 2019Angular Visual Hardnesscited by 29
  25. 2019Regularizing Neural Networks via Minimizing Hyperspherical Energycited by 25
  26. 2019Neural Similarity Learningcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 9; 2023: 7; 2024: 6; 2025: 6; 2026: 3.

35 papers.

Sort:

By area:

  1. 2023Controlling Text-to-Image Diffusion by Orthogonal Finetuningcited by 207
  2. 2021SphereFace2: Binary Classification is All You Need for Deep Face Recognitioncited by 42
  3. 2022The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancycited by 35
  4. 2021SphereFace Revived: Unifying Hyperspherical Face Recognitioncited by 23
  5. 2022Maximum Class Separation as Inductive Bias in One Matrixcited by 21
  6. 2022Hyperspherical Consistency Regularizationcited by 16
  7. 2023Latent SDEs on Homogeneous Spacescited by 16
  8. 2024An Introduction to Deep Survival Analysis Models for Predicting Time-to-Event Outcomescited by 16
  9. 2023Generalizing and Decoupling Neural Collapse via Hyperspherical Uniformity Gapcited by 14
  10. 2021RELAX: Representation Learning Explainabilitycited by 12
  11. 2022Continual Learning by Modeling Intra-Class Variationcited by 12
  12. 2021Iterative Teaching by Label Synthesiscited by 10
23 more
  1. 2022Survival Kernets: Scalable and Interpretable Deep Kernel Survival Analysis with an Accuracy Guaranteecited by 9
  2. 2024FedUV: Uniformity and Variance for Heterogeneous Federated Learningcited by 8
  3. 2025Reparameterized LLM Training via Orthogonal Equivalence Transformationcited by 8
  4. 2025Spectral Scaling Laws in Language Models: How Effectively Do Feed-Forward Networks Use Their Latent Space?cited by 8
  5. 2025Orthogonal Finetuning Made Scalablecited by 7
  6. 2024Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Lossescited by 6
  7. 2024Negative-Free Self-Supervised Gaussian Embedding of Graphscited by 5
  8. 2022Data-Efficient Learning via Minimizing Hyperspherical Energycited by 4
  9. 2022Hyperspherical Quantization: Toward Smaller and More Accurate Modelscited by 4
  10. 2023Hyperspherical Embedding for Point Cloud Completioncited by 4
  11. 2023Pairwise Similarity Learning is SimPLEcited by 4
  12. 2023Novel Categories Discovery Via Constraints on Empirical Prediction Statisticscited by 3
  13. 2026Orthogonal Model Mergingcited by 3
  14. 2026POET-X: Memory-efficient LLM Training by Scaling Orthogonal Transformationcited by 3
  15. 2022Deep Simplex Classifier for Maximizing the Margin in Both Euclidean and Angular Spacescited by 2
  16. 2023A General Framework for Visualizing Embedding Spaces of Neural Survival Analysis Models Based on Angular Informationcited by 2
  17. 2024Enhancing Diversity in Bayesian Deep Learning via Hyperspherical Energy Minimization of CKAcited by 2
  18. 2026Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformationcited by 2
  19. 2025Keep your distance: learning dispersed embeddings on mathbbSmcited by 1
  20. 2025Energy-Regularized Sequential Model Editing on Hyperspherescited by 1
  21. 2022Geometric Regularization from Overparameterizationcited by 0
  22. 2024Towards Better Understanding of Contrastive Sentence Representation Learning: A Unified Paradigm for Gradientcited by 0
  23. 2025Heterogeneous Federated Learning with Prototype Alignment and Upscalingcited by 0