# Hard negative examples are hard, but useful

arXiv: https://arxiv.org/abs/2007.12749 · HTML: https://mlsanity.com/paper/2007.12749/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2020.

## Built on by (most cited first, 30 of 82)

- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [Robust Contrastive Learning Using Negative Samples with Diminished Semantics](https://mlsanity.com/paper/2110.14189/) (2021)
- [Learning Intra-Batch Connections for Deep Metric Learning](https://mlsanity.com/paper/2102.07753/) (2021)
- [Data curation via joint example selection further accelerates multimodal learning](https://mlsanity.com/paper/2406.17711/) (2024)
- [Close, But Not There: Boosting Geographic Distance Sensitivity in Visual Place Recognition](https://mlsanity.com/paper/2407.02422/) (2024)
- [OutfitTransformer: Learning Outfit Representations for Fashion Recommendation](https://mlsanity.com/paper/2204.04812/) (2022)
- [Adaptive Sparse Pairwise Loss for Object Re-Identification](https://mlsanity.com/paper/2303.18247/) (2023)
- [YoChameleon: Personalized Vision and Language Generation](https://mlsanity.com/paper/2504.20998/) (2025)
- [Robust and Decomposable Average Precision for Image Retrieval](https://mlsanity.com/paper/2110.01445/) (2021)
- [Finding Order in Chaos: A Novel Data Augmentation Method for Time Series in Contrastive Learning](https://mlsanity.com/paper/2309.13439/) (2023)
- [Knowledge-enhanced Visual-Language Pretraining for Computational Pathology](https://mlsanity.com/paper/2404.09942/) (2024)
- [Learning by Sorting: Self-supervised Learning with Group Ordering Constraints](https://mlsanity.com/paper/2301.02009/) (2023)
- [Investigating the Limitation of CLIP Models: The Worst-Performing Categories](https://mlsanity.com/paper/2310.03324/) (2023)
- [GRIT-VLP: Grouped Mini-batch Sampling for Efficient Vision and Language Pre-training](https://mlsanity.com/paper/2208.04060/) (2022)
- [Your Negative May not Be True Negative: Boosting Image-Text Matching with False Negative Elimination](https://mlsanity.com/paper/2308.04380/) (2023)
- [Optimization of Rank Losses for Image Retrieval](https://mlsanity.com/paper/2309.08250/) (2023)
- [WildGait: Learning Gait Representations from Raw Surveillance Streams](https://mlsanity.com/paper/2105.05528/) (2021)
- [Confidence-based Visual Dispersal for Few-shot Unsupervised Domain Adaptation](https://mlsanity.com/paper/2309.15575/) (2023)
- [Towards Improved Proxy-based Deep Metric Learning via Data-Augmented Domain Adaptation](https://mlsanity.com/paper/2401.00617/) (2024)
- [Diffusion-based Negative Sampling on Graphs for Link Prediction](https://mlsanity.com/paper/2403.17259/) (2024)
- [ELIP: Enhanced Visual-Language Foundation Models for Image Retrieval](https://mlsanity.com/paper/2502.15682/) (2025)
- [Rethinking Self-Supervision Objectives for Generalizable Coherence Modeling](https://mlsanity.com/paper/2110.07198/) (2021)
- [Relation Preserving Triplet Mining for Stabilising the Triplet Loss in Re-identification Systems](https://mlsanity.com/paper/2110.07933/) (2021)
- [Contrastive Bayesian Analysis for Deep Metric Learning](https://mlsanity.com/paper/2210.04402/) (2022)
- [GeNIe: Generative Hard Negative Images Through Diffusion](https://mlsanity.com/paper/2312.02548/) (2023)
- [Threshold-Consistent Margin Loss for Open-World Deep Metric Learning](https://mlsanity.com/paper/2307.04047/) (2023)
- [SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning](https://mlsanity.com/paper/2510.16416/) (2025)
- [Learning video retrieval models with relevance-aware online mining](https://mlsanity.com/paper/2203.08688/) (2022)
- [Relevance-based Margin for Contrastively-trained Video Retrieval Models](https://mlsanity.com/paper/2204.13001/) (2022)

## Abstract (arXiv)

Triplet loss is an extremely common approach to distance metric learning. Representations of images from the same class are optimized to be mapped closer together in an embedding space than representations of images from different classes. Much work on triplet losses focuses on selecting the most useful triplets of images to consider, with strategies that select dissimilar examples from the same class or similar examples from different classes. The consensus of previous research is that optimizing with the textithardest negative examples leads to bad training behavior. That's a problem -- these hardest negatives are literally the cases where the distance metric fails to capture semantic similarity. In this paper, we characterize the space of triplets and derive why hard negatives make triplet loss training fail. We offer a simple fix to the loss function and show that, with this fix, optimizing with hard negative examples becomes feasible. This leads to more generalizable features, and image retrieval results that outperform state of the art for datasets with high intra-class variance.