# Adversarial Contrastive Estimation

arXiv: https://arxiv.org/abs/1805.03642 · HTML: https://mlsanity.com/paper/1805.03642/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2018.

## Built on by (most cited first, 23 of 23)

- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)
- [Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capability](https://mlsanity.com/paper/2411.19943/) (2024)
- [Negative Data Augmentation](https://mlsanity.com/paper/2102.05113/) (2021)
- [Energy-Based Imitation Learning](https://mlsanity.com/paper/2004.09395/) (2020)
- [Seeking Patterns, Not just Memorizing Procedures: Contrastive Learning for Solving Math Word Problems](https://mlsanity.com/paper/2110.08464/) (2021)
- [Does Negative Sampling Matter? A Review with Insights into its Theory and Applications](https://mlsanity.com/paper/2402.17238/) (2024)
- [Boosting Semi-supervised Image Segmentation with Global and Local Mutual Information Regularization](https://mlsanity.com/paper/2103.04813/) (2021)
- [Contrastive Attraction and Contrastive Repulsion for Representation Learning](https://mlsanity.com/paper/2105.03746/) (2021)
- [Towards User-Driven Neural Machine Translation](https://mlsanity.com/paper/2106.06200/) (2021)
- [Self-supervised Context-aware Style Representation for Expressive Speech Synthesis](https://mlsanity.com/paper/2206.12559/) (2022)
- [Revisiting Energy Based Models as Policies: Ranking Noise Contrastive Estimation and Interpolating Energy Models](https://mlsanity.com/paper/2309.05803/) (2023)
- [Adversarial Contrastive Pre-training for Protein Sequences](https://mlsanity.com/paper/2102.00466/) (2021)
- [Identifiable Energy-based Representations: An Application to Estimating Heterogeneous Causal Effects](https://mlsanity.com/paper/2108.03039/) (2021)
- [Moment Matching Denoising Gibbs Sampling](https://mlsanity.com/paper/2305.11650/) (2023)
- [Gradient-Guided Importance Sampling for Learning Binary Energy-Based Models](https://mlsanity.com/paper/2210.05782/) (2022)
- [Learning Unnormalized Statistical Models via Compositional Optimization](https://mlsanity.com/paper/2306.07485/) (2023)
- [Data-Driven Stochastic Motion Evaluation and Optimization with Image by Spatially-Aligned Temporal Encoding](https://mlsanity.com/paper/2302.05041/) (2023)
- [ContrastNER: Contrastive-based Prompt Tuning for Few-shot NER](https://mlsanity.com/paper/2305.17951/) (2023)
- [C-LEAD: Contrastive Learning for Enhanced Adversarial Defense](https://mlsanity.com/paper/2510.27249/) (2025)
- [TFDPM: Attack detection for cyber-physical systems with diffusion probabilistic models](https://mlsanity.com/paper/2112.10774/) (2021)
- [Self-Adapting Noise-Contrastive Estimation for Energy-Based Models](https://mlsanity.com/paper/2211.02650/) (2022)
- [Adversarially Contrastive Estimation of Conditional Neural Processes](https://mlsanity.com/paper/2303.13004/) (2023)
- [FluentEditor2: Text-based Speech Editing by Modeling Multi-Scale Acoustic and Prosody Consistency](https://mlsanity.com/paper/2410.03719/) (2024)

## Abstract (arXiv)

Learning by contrasting positive and negative samples is a general strategy adopted by many methods. Noise contrastive estimation (NCE) for word embeddings and translating embeddings for knowledge graphs are examples in NLP employing this approach. In this work, we view contrastive learning as an abstraction of all such methods and augment the negative sampler into a mixture distribution containing an adversarially learned sampler. The resulting adaptive sampler finds harder negative examples, which forces the main model to learn a better representation of the data. We evaluate our proposal on learning word embeddings, order embeddings and knowledge graph embeddings and observe both faster convergence and improved results on multiple metrics.