# Deep Hyperspherical Learning

arXiv: https://arxiv.org/abs/1711.03189 · HTML: https://mlsanity.com/paper/1711.03189/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2017.

## Built on by (most cited first, 30 of 55)

- [Controlling Text-to-Image Diffusion by Orthogonal Finetuning](https://mlsanity.com/paper/2306.07280/) (2023)
- [Towards Principled Disentanglement for Domain Generalization](https://mlsanity.com/paper/2111.13839/) (2021)
- [B-cos Networks: Alignment is All We Need for Interpretability](https://mlsanity.com/paper/2205.10268/) (2022)
- [nGPT: Normalized Transformer with Representation Learning on the Hypersphere](https://mlsanity.com/paper/2410.01131/) (2024)
- [Orthogonal Projection Loss](https://mlsanity.com/paper/2103.14021/) (2021)
- [SphereFace2: Binary Classification is All You Need for Deep Face Recognition](https://mlsanity.com/paper/2108.01513/) (2021)
- [Hyperspherical Normalization for Scalable Deep Reinforcement Learning](https://mlsanity.com/paper/2502.15280/) (2025)
- [Learning with Hyperspherical Uniformity](https://mlsanity.com/paper/2103.01649/) (2021)
- [B-cos Alignment for Inherently Interpretable CNNs and Vision Transformers](https://mlsanity.com/paper/2306.10898/) (2023)
- [VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Models](https://mlsanity.com/paper/2412.01095/) (2024)
- [Learning by Turning: Neural Architecture Aware Optimisation](https://mlsanity.com/paper/2102.07227/) (2021)
- [SphereFace Revived: Unifying Hyperspherical Face Recognition](https://mlsanity.com/paper/2109.05565/) (2021)
- [Maximum Class Separation as Inductive Bias in One Matrix](https://mlsanity.com/paper/2206.08704/) (2022)
- [Hyperspherical Consistency Regularization](https://mlsanity.com/paper/2206.00845/) (2022)
- [Latent SDEs on Homogeneous Spaces](https://mlsanity.com/paper/2306.16248/) (2023)
- [Curved Geometric Networks for Visual Anomaly Recognition](https://mlsanity.com/paper/2208.01188/) (2022)
- [Generalizing and Decoupling Neural Collapse via Hyperspherical Uniformity Gap](https://mlsanity.com/paper/2303.06484/) (2023)
- [Stronger Normalization-Free Transformers](https://mlsanity.com/paper/2512.10938/) (2025)
- [Continual Learning by Modeling Intra-Class Variation](https://mlsanity.com/paper/2210.05398/) (2022)
- [Iterative Teaching by Label Synthesis](https://mlsanity.com/paper/2110.14432/) (2021)
- [Learning to Disentangle Scenes for Person Re-identification](https://mlsanity.com/paper/2111.05476/) (2021)
- [Raising the Bar in Graph OOD Generalization: Invariant Learning Beyond Explicit Environment Modeling](https://mlsanity.com/paper/2502.10706/) (2025)
- [Reparameterized LLM Training via Orthogonal Equivalence Transformation](https://mlsanity.com/paper/2506.08001/) (2025)
- [Spectral Scaling Laws in Language Models: How Effectively Do Feed-Forward Networks Use Their Latent Space?](https://mlsanity.com/paper/2510.00537/) (2025)
- [Angular Gap: Reducing the Uncertainty of Image Difficulty through Model Calibration](https://mlsanity.com/paper/2207.08525/) (2022)
- [Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks](https://mlsanity.com/paper/2305.15912/) (2023)
- [Efficient Low-Resolution Face Recognition via Bridge Distillation](https://mlsanity.com/paper/2409.11786/) (2024)
- [Data-Efficient Learning via Minimizing Hyperspherical Energy](https://mlsanity.com/paper/2206.15204/) (2022)
- [Hyperspherical Quantization: Toward Smaller and More Accurate Models](https://mlsanity.com/paper/2212.12653/) (2022)
- [Distortion-Disentangled Contrastive Learning](https://mlsanity.com/paper/2303.05066/) (2023)

## Abstract (arXiv)

Convolution as inner product has been the founding basis of convolutional neural networks (CNNs) and the key to end-to-end visual representation learning. Benefiting from deeper architectures, recent CNNs have demonstrated increasingly strong representation abilities. Despite such improvement, the increased depth and larger parameter space have also led to challenges in properly training a network. In light of such challenges, we propose hyperspherical convolution (SphereConv), a novel learning framework that gives angular representations on hyperspheres. We introduce SphereNet, deep hyperspherical convolution networks that are distinct from conventional inner product based convolutional networks. In particular, SphereNet adopts SphereConv as its basic convolution operator and is supervised by generalized angular softmax loss - a natural loss formulation under SphereConv. We show that SphereNet can effectively encode discriminative representation and alleviate training difficulty, leading to easier optimization, faster convergence and comparable (even better) classification accuracy over convolutional counterparts. We also provide some theoretical insights for the advantages of learning on hyperspheres. In addition, we introduce the learnable SphereConv, i.e., a natural improvement over prefixed SphereConv, and SphereNorm, i.e., hyperspherical learning as a normalization method. Experiments have verified our conclusions.