# Softmax Dissection: Towards Understanding Intra- and Inter-class Objective for Embedding Learning

arXiv: https://arxiv.org/abs/1908.01281 · HTML: https://mlsanity.com/paper/1908.01281/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Aug 2019.

## Built on by (most cited first, 10 of 10)

- [Killing Two Birds with One Stone:Efficient and Robust Training of Face Recognition CNNs by Partial FC](https://mlsanity.com/paper/2203.15565/) (2022)
- [SFace: Sigmoid-Constrained Hypersphere Loss for Robust Face Recognition](https://mlsanity.com/paper/2205.12010/) (2022)
- [Racial Bias within Face Recognition: A Survey](https://mlsanity.com/paper/2305.00817/) (2023)
- [Training and challenging models for text-guided fashion image retrieval](https://mlsanity.com/paper/2204.11004/) (2022)
- [Prototype Memory for Large-scale Face Representation Learning](https://mlsanity.com/paper/2105.02103/) (2021)
- [Mitigating Catastrophic Forgetting in the Incremental Learning of Medical Images](https://mlsanity.com/paper/2504.20033/) (2025)
- [Deep Boosting Multi-Modal Ensemble Face Recognition with Sample-Level Weighting](https://mlsanity.com/paper/2308.09234/) (2023)
- [Quality-Aware Prototype Memory for Face Representation Learning](https://mlsanity.com/paper/2311.07734/) (2023)
- [Realigned Softmax Warping for Deep Metric Learning](https://mlsanity.com/paper/2408.15656/) (2024)
- [DAAL: Density-Aware Adaptive Line Margin Loss for Multi-Modal Deep Metric Learning](https://mlsanity.com/paper/2410.05438/) (2024)

## Abstract (arXiv)

The softmax loss and its variants are widely used as objectives for embedding learning, especially in applications like face recognition. However, the intra- and inter-class objectives in the softmax loss are entangled, therefore a well-optimized inter-class objective leads to relaxation on the intra-class objective, and vice versa. In this paper, we propose to dissect the softmax loss into independent intra- and inter-class objective (D-Softmax). With D-Softmax as objective, we can have a clear understanding of both the intra- and inter-class objective, therefore it is straightforward to tune each part to the best state. Furthermore, we find the computation of the inter-class objective is redundant and propose two sampling-based variants of D-Softmax to reduce the computation cost. Training with regular-scale data, experiments in face verification show D-Softmax is favorably comparable to existing losses such as SphereFace and ArcFace. Training with massive-scale data, experiments show the fast variants of D-Softmax significantly accelerates the training process (such as 64x) with only a minor sacrifice in performance, outperforming existing acceleration methods of softmax in terms of both performance and efficiency.