# Negative Margin Matters: Understanding Margin in Few-shot Classification

arXiv: https://arxiv.org/abs/2003.12060 · HTML: https://mlsanity.com/paper/2003.12060/

## Key facts

- Cited by 129 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2020.

## Built on by (most cited first, 30 of 129)

- [Free Lunch for Few-shot Learning: Distribution Calibration](https://mlsanity.com/paper/2101.06395/) (2021)
- [Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2102.03526/) (2021)
- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Forward Compatible Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2203.06953/) (2022)
- [Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer](https://mlsanity.com/paper/2108.03032/) (2021)
- [Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2302.03004/) (2023)
- [Beyond Max-Margin: Class Margin Equilibrium for Few-shot Object Detection](https://mlsanity.com/paper/2103.04612/) (2021)
- [Relational Embedding for Few-Shot Classification](https://mlsanity.com/paper/2108.09666/) (2021)
- [Rectifying the Shortcut Learning of Background for Few-Shot Learning](https://mlsanity.com/paper/2107.07746/) (2021)
- [Boosting the Generalization Capability in Cross-Domain Few-shot Learning via Noise-enhanced Supervised Autoencoder](https://mlsanity.com/paper/2108.05028/) (2021)
- [Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification](https://mlsanity.com/paper/2204.04567/) (2022)
- [Few-Shot Class-Incremental Learning via Training-Free Prototype Calibration](https://mlsanity.com/paper/2312.05229/) (2023)
- [Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigation](https://mlsanity.com/paper/2210.04524/) (2022)
- [Orthogonal Projection Loss](https://mlsanity.com/paper/2103.14021/) (2021)
- [Learning Dynamic Alignment via Meta-filter for Few-shot Learning](https://mlsanity.com/paper/2103.13582/) (2021)
- [Generating Representative Samples for Few-Shot Classification](https://mlsanity.com/paper/2205.02918/) (2022)
- [Revisiting the Transferability of Supervised Pretraining: an MLP Perspective](https://mlsanity.com/paper/2112.00496/) (2021)
- [Adversarial Feature Augmentation for Cross-domain Few-shot Classification](https://mlsanity.com/paper/2208.11021/) (2022)
- [PML: Progressive Margin Loss for Long-tailed Age Classification](https://mlsanity.com/paper/2103.02140/) (2021)
- [Boosting Few-shot Fine-grained Recognition with Background Suppression and Foreground Alignment](https://mlsanity.com/paper/2210.01439/) (2022)
- [Few-shot Action Recognition with Prototype-centered Attentive Learning](https://mlsanity.com/paper/2101.08085/) (2021)
- [Binocular Mutual Learning for Improving Few-shot Classification](https://mlsanity.com/paper/2108.12104/) (2021)
- [Integrative Few-Shot Learning for Classification and Segmentation](https://mlsanity.com/paper/2203.15712/) (2022)
- [Matching Feature Sets for Few-Shot Image Classification](https://mlsanity.com/paper/2204.00949/) (2022)
- [HyperShot: Few-Shot Learning by Kernel HyperNetworks](https://mlsanity.com/paper/2203.11378/) (2022)
- [Learngene: Inheriting Condensed Knowledge from the Ancestry Model to Descendant Models](https://mlsanity.com/paper/2305.02279/) (2023)
- [Distilling Self-Supervised Vision Transformers for Weakly-Supervised Few-Shot Classification & Segmentation](https://mlsanity.com/paper/2307.03407/) (2023)
- [Deep Metric Learning for Few-Shot Image Classification: A Review of Recent Developments](https://mlsanity.com/paper/2105.08149/) (2021)
- [Anti-aliasing Semantic Reconstruction for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2106.00184/) (2021)
- [Few-shot Learning with Noisy Labels](https://mlsanity.com/paper/2204.05494/) (2022)

## Abstract (arXiv)

This paper introduces a negative margin loss to metric learning based few-shot learning methods. The negative margin loss significantly outperforms regular softmax loss, and achieves state-of-the-art accuracy on three standard few-shot classification benchmarks with few bells and whistles. These results are contrary to the common practice in the metric learning field, that the margin is zero or positive. To understand why the negative margin loss performs well for the few-shot classification, we analyze the discriminability of learned features w.r.t different margins for training and novel classes, both empirically and theoretically. We find that although negative margin reduces the feature discriminability for training classes, it may also avoid falsely mapping samples of the same novel class to multiple peaks or clusters, and thus benefit the discrimination of novel classes. Code is available at https://github.com/bl0/negative-margin.few-shot.