mlsanityData to 7 Oct 2026

Data Augmentation of Contrastive Learning is Estimating Positive-incentive Noise

arXiv 2408.09929 ↗First posted 19 Aug 2024cs.LG cs.CVcited by 25

Not on the map. Why?

Abstract (arXiv)

Inspired by the idea of Positive-incentive Noise (Pi-Noise or π-Noise) that aims at learning the reliable noise beneficial to tasks, we scientifically investigate the connection between contrastive learning and π-noise in this paper. By converting the contrastive loss to an auxiliary Gaussian distribution to quantitatively measure the difficulty of the specific contrastive model under the information theory framework, we properly define the task entropy, the core concept of π-noise, of contrastive learning. It is further proved that the predefined data augmentation in the standard contrastive learning paradigm can be regarded as a kind of point estimation of π-noise. Inspired by the theoretical study, a framework that develops a π-noise generator to learn the beneficial noise (instead of estimation) as data augmentations for contrast is proposed. The designed framework can be applied to diverse types of data and is also completely compatible with the existing contrastive models. From the visualization, we surprisingly find that the proposed method successfully learns effective augmentations. Our code is available at https://github.com/hyzhang98/PiNDA.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 22 of its 38 references that are in the map. First posted on arXiv 19 Aug 2024.

Builds on

22 of 38 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2023Segment Anythingcited by 8,158
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  10. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  11. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  12. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
10 more
  1. 2020Contrastive Learning with Hard Negative Samplescited by 619
  2. 2020Graph Contrastive Learning with Adaptive Augmentationcited by 519
  3. 2013Introducing LETOR 4.0 Datasetscited by 120
  4. 2020Contrastive Learning with Adversarial Examplescited by 91
  5. 2021Multi-view Contrastive Graph Clusteringcited by 76
  6. 2020Towards Domain-Agnostic Contrastive Learningcited by 75
  7. 2022Positive-incentive Noisecited by 39
  8. 2023Variational Positive-incentive Noise: How Noise Benefits Modelscited by 38
  9. 2022An Empirical Study on Disentanglement of Negative-free Contrastive Learningcited by 12
  10. 2020Generalized Clustering and Multi-Manifold Learning with Geometric Structure Preservationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 17; 2026: 4.

25 papers.

Sort:

By area:

  1. 2024Enhance Vision-Language Alignment with Noisecited by 34
  2. 2025Learn Beneficial Noise as Graph Augmentationcited by 15
  3. 2025NFIG: Multi-Scale Autoregressive Image Generation via Frequency Orderingcited by 14
  4. 2025Mixture of Noise for Pre-Trained Model-Based Class-Incremental Learningcited by 14
  5. 2025Adv-CPG: A Customized Portrait Generation Framework with Facial Adversarial Attackscited by 13
  6. 2025Object-AVEdit: An Object-level Audio-Visual Editing Modelcited by 12
  7. 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advancescited by 11
  8. 2025Explore How to Inject Beneficial Noise in MLLMscited by 10
  9. 2025MR-UIE: Multi-Perspective Reasoning with Reinforcement Learning for Universal Information Extractioncited by 6
  10. 2025Laytrol: Preserving Pretrained Knowledge in Layout Control for Multimodal Diffusion Transformerscited by 6
  11. 2024Why Does Dropping Edges Usually Outperform Adding Edges in Graph Contrastive Learning?cited by 5
  12. 2025AnchorFormer: Differentiable Anchor Attention for Efficient Vision Transformercited by 3
13 more
  1. 2025TableZoomer: A Collaborative Agent Framework for Large-scale Table Question Answeringcited by 3
  2. 2025GRPO-RM: Fine-Tuning Representation Models via GRPO-Driven Reinforcement Learningcited by 2
  3. 2024Edge Contrastive Learning: An Augmentation-Free Graph Contrastive Learning Modelcited by 1
  4. 2025ERF-BA-TFD+: A Multimodal Model for Audio-Visual Deepfake Detectioncited by 1
  5. 2026Safeguarding Text-to-Image Generative Models Against Unauthorized Knowledge Distillationcited by 1
  6. 2024Multi-Task Curriculum Graph Contrastive Learning with Clustering Entropy Guidancecited by 0
  7. 2025Real-Time Crowd Counting for Embedded Systems with Lightweight Architecturecited by 0
  8. 2025CoLM: Collaborative Large Models via A Client-Server Paradigmcited by 0
  9. 2025How Noise Benefits AI-generated Image Detectioncited by 0
  10. 2025Multi-Intent Spoken Language Understanding: Methods, Trends, and Challengescited by 0
  11. 2026Noise-Robust Tiny Object Localization with Flowscited by 0
  12. 2026Miles: Metric Learning with Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learningcited by 0
  13. 2026On-Policy Distillation for Vision-Language Model Adaptation, an Effective Paradigm on Low-Quality Multimodal Datacited by 0