mlsanityData to 7 Oct 2026

Contrastive Representation Learning with Trainable Augmentation Channel

arXiv 2111.07679 ↗First posted 15 Nov 2021stat.ML cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

In contrastive representation learning, data representation is trained so that it can classify the image instances even when the images are altered by augmentations. However, depending on the datasets, some augmentations can damage the information of the images beyond recognition, and such augmentations can result in collapsed representations. We present a partial solution to this problem by formalizing a stochastic encoding process in which there exist a tug-of-war between the data corruption introduced by the augmentations and the information preserved by the encoder. We show that, with the infoMax objective based on this framework, we can learn a data-dependent distribution of augmentations to avoid the collapse of the representation.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 18 of its 21 references that are in the map. First posted on arXiv 15 Nov 2021.

Builds on

18 of 21 references.

References · 18

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  3. 2019Contrastive Multiview Codingcited by 1,297
  4. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  5. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  6. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  7. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  8. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  9. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  10. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  11. 2019On Mutual Information Maximization for Representation Learningcited by 315
  12. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
6 more
  1. 2017Learning Discrete Representations via Information Maximizing Self-Augmented Trainingcited by 149
  2. 2019Faster AutoAugment: Learning Augmentation Strategies using Backpropagationcited by 96
  3. 2020On Mutual Information in Contrastive Learning for Visual Representationscited by 54
  4. 2020Viewmaker Networks: Learning Views for Unsupervised Representation Learningcited by 48
  5. 2020SelfAugment: Automatic Augmentation Policies for Self-Supervised Learningcited by 32
  6. 2019Noise Regularization for Conditional Density Estimationcited by 23

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022Adversarial Masking for Self-Supervised Learningcited by 78