A Coding-Theoretic Analysis of Hyperspherical Prototypical Learning Geometry
Not on the map. Why?
Abstract (arXiv)
Hyperspherical Prototypical Learning (HPL) is a supervised approach to representation learning that designs class prototypes on the unit hypersphere. The prototypes bias the representations to class separation in a scale invariant and known geometry. Previous approaches to HPL have either of the following shortcomings: (i) they follow an unprincipled optimisation procedure; or (ii) they are theoretically sound, but are constrained to only one possible latent dimension. In this paper, we address both shortcomings. To address (i), we present a principled optimisation procedure whose solution we show is optimal. To address (ii), we construct well-separated prototypes in a wide range of dimensions using linear block codes. Additionally, we give a full characterisation of the optimal prototype placement in terms of achievable and converse bounds, showing that our proposed methods are near-optimal.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 17 of its 35 references that are in the map. First posted on arXiv 10 Jul 2024.
Builds on
17 of 35 references.
References · 17
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2017Prototypical Networks for Few-shot Learningcited by 3,015
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
- 2020Supervised Contrastive Learningcited by 2,829
- 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
- 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
- 2019Contrastive Multiview Codingcited by 1,301
- 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
- 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 275
- 2017NormFace: L2 Hypersphere Embedding for Face Verificationcited by 237
- 2018Hyperspherical Variational Auto-Encoderscited by 189
5 more
- 2019Hyperspherical Prototype Networkscited by 86
- 2018Spherical Latent Spaces for Stable Variational Autoencoderscited by 67
- 2017von Mises-Fisher Mixture Model-based Deep learning: Application to Face Verificationcited by 46
- 2022Maximum Class Separation as Inductive Bias in One Matrixcited by 21
- 2023The Role of Entropy and Reconstruction in Multi-View Self-Supervised Learningcited by 13
Built on by
Citing papers per year: 2026: 1.
1 paper.