mlsanityData to 7 Oct 2026

von Mises-Fisher Loss: An Exploration of Embedding Geometries for Supervised Learning

arXiv 2103.15718 ↗First posted 29 Mar 2021cs.LG cs.CVcited by 42

Not on the map. Why?

Abstract (arXiv)

Recent work has argued that classification losses utilizing softmax cross-entropy are superior not only for fixed-set classification tasks, but also by outperforming losses developed specifically for open-set tasks including few-shot learning and retrieval. Softmax classifiers have been studied using different embedding geometries -- Euclidean, hyperbolic, and spherical -- and claims have been made about the superiority of one or another, but they have not been systematically compared with careful controls. We conduct an empirical investigation of embedding geometry on softmax losses for a variety of fixed-set classification and image retrieval tasks. An interesting property observed for the spherical losses lead us to propose a probabilistic classifier based on the von Mises-Fisher distribution, and we show that it is competitive with state-of-the-art methods while producing improved out-of-the-box calibration. We provide guidance regarding the trade-offs between losses and how to choose among them.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 53 of its 82 references that are in the map. First posted on arXiv 29 Mar 2021.

Builds on

53 of 82 references.

References · 53

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  6. 2016Matching Networks for One Shot Learningcited by 2,246
  7. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  8. 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 968
  9. 2016Deep Variational Information Bottleneckcited by 915
  10. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 733
  11. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 674
  12. 2015Deep Metric Learning via Lifted Structured Feature Embeddingcited by 449
38 more
  1. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  2. 2020Circle Loss: A Unified Perspective of Pair Similarity Optimizationcited by 402
  3. 2018Hyperbolic Neural Networkscited by 393
  4. 2019Multi-Similarity Loss with General Pair Weighting for Deep Metric Learningcited by 368
  5. 2020A Metric Learning Reality Checkcited by 317
  6. 2019Verified Uncertainty Calibrationcited by 307
  7. 2017Sampling Matters in Deep Embedding Learningcited by 285
  8. 2018Additive Margin Softmax for Face Verificationcited by 275
  9. 2017NormFace: L2 Hypersphere Embedding for Face Verificationcited by 237
  10. 2017No Fuss Distance Metric Learning using Proxiescited by 225
  11. 2019Hyperbolic Image Embeddingscited by 219
  12. 2018Hyperspherical Variational Auto-Encoderscited by 188
  13. 2019Probabilistic Face Embeddingscited by 171
  14. 2019SoftTriple Loss: Deep Metric Learning Without Triplet Samplingcited by 163
  15. 2016Learning a Deep Embedding Model for Zero-Shot Learningcited by 153
  16. 2020Data Uncertainty Learning in Face Recognitioncited by 134
  17. 2017Deep Metric Learning with Angular Losscited by 107
  18. 2020ProxyNCA++: Revisiting and Revitalizing Proxy Neighborhood Component Analysiscited by 107
  19. 2018Learning Deep Disentangled Embeddings with the F-Statistic Losscited by 101
  20. 2020A unifying mutual information view of metric learning: cross-entropy vs. pairwise lossescited by 100
  21. 2020Mitigating Bias in Calibration Error Estimationcited by 100
  22. 2019Infinite Mixture Prototypes for Few-Shot Learningcited by 99
  23. 2017L2-constrained Softmax Loss for Discriminative Face Verificationcited by 96
  24. 2019Hyperspherical Prototype Networkscited by 85
  25. 2019Ranked List Loss for Deep Metric Learningcited by 77
  26. 2017Learning to Remember Rare Eventscited by 63
  27. 2016Learning Deep Embeddings with Histogram Losscited by 52
  28. 2018Modeling Uncertainty with Hedged Instance Embeddingcited by 52
  29. 2017Few-Shot Learning Through an Information Retrieval Lenscited by 51
  30. 2017von Mises-Fisher Mixture Model-based Deep learning: Application to Face Verificationcited by 45
  31. 2015Metric Learning with Adaptive Density Discriminationcited by 40
  32. 2019Signal-to-Noise Ratio: A Robust Distance Metric for Deep Metric Learningcited by 32
  33. 2019Angular Visual Hardnesscited by 29
  34. 2015An Exploration of Softmax Alternatives Belonging to the Spherical Loss Familycited by 27
  35. 2018Von Mises-Fisher Loss for Training Sequence to Sequence Models with Continuous Outputscited by 26
  36. 2017Gaussian Prototypical Networks for Few-Shot Learning on Omniglotcited by 20
  37. 2015Targeting Ultimate Accuracy: Face Recognition via Deep Embeddingcited by 18
  38. 2018Directional Statistics-based Deep Metric Learning for Image Classification and Retrievalcited by 16

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 8; 2024: 10; 2025: 8; 2026: 8.

42 papers.

Sort:

By area:

  1. 2024Probabilistic Contrastive Learning for Long-Tailed Visual Recognitioncited by 42
  2. 2024Transductive Zero-Shot and Few-Shot CLIPcited by 31
  3. 2023Probabilistic Contrastive Learning Recovers the Correct Aleatoric Uncertainty of Ambiguous Inputscited by 28
  4. 2023Geometric Autoencoders -- What You See is What You Decodecited by 23
  5. 2022Representation Uncertainty in Self-Supervised Learning as Variational Inferencecited by 22
  6. 2022A Non-isotropic Probabilistic Take on Proxy-based Deep Metric Learningcited by 19
  7. 2023URL: A Representation Learning Benchmark for Transferable Uncertainty Estimatescited by 17
  8. 2022Boosting Semi-Supervised Semantic Segmentation with Probabilistic Representationscited by 10
  9. 2024Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modelingcited by 10
  10. 2025On the Importance of Embedding Norms in Self-Supervised Learningcited by 9
  11. 2022Probabilistic Embeddings Revisitedcited by 8
  12. 2022Shedding a PAC-Bayesian Light on Adaptive Sliced-Wasserstein Distancescited by 7
30 more
  1. 2023Introspective Deep Metric Learningcited by 7
  2. 2023Variational Classificationcited by 5
  3. 2023Factorizers for Distributed Sparse Block Codescited by 4
  4. 2024Your Classifier Can Be Secretly a Likelihood-Based OOD Detectorcited by 4
  5. 2025Exploiting the Asymmetric Uncertainty Structure of Pre-trained VLMs on the Unit Hyperspherecited by 4
  6. 2022Magnitude-aware Probabilistic Speaker Embeddingscited by 3
  7. 2022ScaleFace: Uncertainty-aware Deep Metric Learningcited by 3
  8. 2023Few-shot Classification via Ensemble Learning with Multi-Order Statisticscited by 3
  9. 2024PRCL: Probabilistic Representation Contrastive Learning for Semi-Supervised Semantic Segmentationcited by 3
  10. 2024Context-Aware Multimodal Pretrainingcited by 3
  11. 2025Harmonizing Geometry and Uncertainty: Diffusion with Hyperspherescited by 3
  12. 2025vMFCoOp: Towards Equilibrium on a Unified Hyperspherical Manifold for Prompting Biomedical VLMscited by 3
  13. 2023The Unreasonable Effectiveness of Random Target Embeddings for Continuous-Output Neural Machine Translationcited by 2
  14. 2024Distribution Guidance Network for Weakly Supervised Point Cloud Semantic Segmentationcited by 2
  15. 2022An Empirical Study on Clustering Pretrained Embeddings: Is Deep Strictly Better?cited by 1
  16. 2024Dynamic Feature Learning and Matching for Class-Incremental Learningcited by 1
  17. 2025SphOR: A Representation Learning Perspective on Open-set Recognition for Identifying Unknown Classes in Deep Learning Modelscited by 1
  18. 2025Spherical Cauchy Variational Autoencoders: Heavy Angular Tails and Exact KL Evaluationcited by 1
  19. 2025Beyond Distributions: Geometric Action Control for Continuous Reinforcement Learningcited by 1
  20. 2026Context Sensitivity Improves Human-Machine Visual Alignmentcited by 1
  21. 2026FedLAS: Feature-Modulated Bidirectional Label Smoothing for Neural Network Calibrationcited by 1
  22. 2024Probabilistic Contrastive Learning with Explicit Concentration on the Hyperspherecited by 0
  23. 2024Reducing Semantic Ambiguity In Domain Adaptive Semantic Segmentation Via Probabilistic Prototypical Pixel Contrastcited by 0
  24. 2025Divergence-Based Similarity Function for Multi-View Contrastive Learningcited by 0
  25. 2026Channel-Adaptive Edge AI: Maximizing Inference Throughput by Adapting Computational Complexity to Channel Statescited by 0
  26. 2026GeoFlow: Real-Time Fine-Grained Cross-View Geolocalization via Iterative Flow Predictioncited by 0
  27. 2026When Features Become Instances: Inverted Contrastive Learning for Unsupervised Feature Selectioncited by 0
  28. 2026Predicting build orientation for SLM dental parts: a comparison of rotation representations and direct vector regressioncited by 0
  29. 2026How Accurate Is Accurate Enough?cited by 0
  30. 2026Same Loss, Different Gradientscited by 0