# Large-Margin Softmax Loss for Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1612.02295 · HTML: https://mlsanity.com/paper/1612.02295/

## Key facts

- Cited by 296 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2016.

## Built on by (most cited first, 30 of 296)

- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)
- [WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2103.04098/) (2021)
- [Unified Contrastive Learning in Image-Text-Label Space](https://mlsanity.com/paper/2204.03610/) (2022)
- [Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2102.03526/) (2021)
- [ElasticFace: Elastic Margin Loss for Deep Face Recognition](https://mlsanity.com/paper/2109.09416/) (2021)
- [Rethinking Semantic Segmentation: A Prototype View](https://mlsanity.com/paper/2203.15102/) (2022)
- [Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection](https://mlsanity.com/paper/2103.09096/) (2021)
- [Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification](https://mlsanity.com/paper/2103.14267/) (2021)
- [Forward Compatible Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2203.06953/) (2022)
- [Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypes](https://mlsanity.com/paper/2111.15000/) (2021)
- [Low-Cost High-Power Membership Inference Attacks](https://mlsanity.com/paper/2312.03262/) (2023)
- [SynFace: Face Recognition with Synthetic Data](https://mlsanity.com/paper/2108.07960/) (2021)
- [Neural Collapse: A Review on Modelling Principles and Generalization](https://mlsanity.com/paper/2206.04041/) (2022)
- [GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models](https://mlsanity.com/paper/2210.02025/) (2022)
- [Debiased Learning from Naturally Imbalanced Pseudo-Labels](https://mlsanity.com/paper/2201.01490/) (2022)
- [Are All Losses Created Equal: A Neural Collapse Perspective](https://mlsanity.com/paper/2210.02192/) (2022)
- [PatchNet: A Simple Face Anti-Spoofing Framework via Fine-Grained Patch Recognition](https://mlsanity.com/paper/2203.14325/) (2022)
- [Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2304.00426/) (2023)
- [Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regularized Fine-Tuning](https://mlsanity.com/paper/2102.06605/) (2021)
- [Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold](https://mlsanity.com/paper/2209.09211/) (2022)
- [A Comprehensive Survey on Deep Gait Recognition: Algorithms, Datasets and Challenges](https://mlsanity.com/paper/2206.13732/) (2022)
- [Orthogonal Projection Loss](https://mlsanity.com/paper/2103.14021/) (2021)
- [Deep Learning for Micro-expression Recognition: A Survey](https://mlsanity.com/paper/2107.02823/) (2021)
- [SphereFace2: Binary Classification is All You Need for Deep Face Recognition](https://mlsanity.com/paper/2108.01513/) (2021)
- [LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds](https://mlsanity.com/paper/2210.08064/) (2022)
- [Partner-Assisted Learning for Few-Shot Image Classification](https://mlsanity.com/paper/2109.07607/) (2021)
- [Learning with Hyperspherical Uniformity](https://mlsanity.com/paper/2103.01649/) (2021)
- [The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy](https://mlsanity.com/paper/2203.06345/) (2022)
- [Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuning](https://mlsanity.com/paper/2308.09303/) (2023)
- [Analyzing Overfitting under Class Imbalance in Neural Networks for Image Segmentation](https://mlsanity.com/paper/2102.10365/) (2021)

## Abstract (arXiv)

Cross-entropy loss together with softmax is arguably one of the most common used supervision components in convolutional neural networks (CNNs). Despite its simplicity, popularity and excellent performance, the component does not explicitly encourage discriminative learning of features. In this paper, we propose a generalized large-margin softmax (L-Softmax) loss which explicitly encourages intra-class compactness and inter-class separability between learned features. Moreover, L-Softmax not only can adjust the desired margin but also can avoid overfitting. We also show that the L-Softmax loss can be optimized by typical stochastic gradient descent. Extensive experiments on four benchmark datasets demonstrate that the deeply-learned features with L-softmax loss become more discriminative, hence significantly boosting the performance on a variety of visual classification and verification tasks.