mlsanityData to 14 Oct 2026

On minimal variations for unsupervised representation learning

arXiv 2211.03782 ↗First posted 7 Nov 2022cs.LG cs.AI stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

Unsupervised representation learning aims at describing raw data efficiently to solve various downstream tasks. It has been approached with many techniques, such as manifold learning, diffusion maps, or more recently self-supervised learning. Those techniques are arguably all based on the underlying assumption that target functions, associated with future downstream tasks, have low variations in densely populated regions of the input space. Unveiling minimal variations as a guiding principle behind unsupervised representation learning paves the way to better practical guidelines for self-supervised learning algorithms.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 12 of its 31 references that are in the map. First posted on arXiv 7 Nov 2022.

Builds on

12 of 31 references.

References · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  6. 2017Improved Training of Wasserstein GANscited by 2,235
  7. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  8. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  9. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 313
  10. 2022Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methodscited by 149
  11. 2019Robust Learning with Jacobian Regularizationcited by 129
  12. 2020Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learningcited by 14

Built on by

Cited by year

Citing papers per year: 2023: 5; 2024: 1.

6 papers.

  1. 2023The SSL Interplay: Augmentations, Inductive Bias, and Generalizationcited by 40
  2. 2023Active Self-Supervised Learning: A Few Low-Cost Relationships Are All You Needcited by 9
  3. 2024Occam's Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?cited by 8
  4. 2023The Galerkin method beats Graph-Based Approaches for Spectral Algorithmscited by 6
  5. 2023Kernelized Diffusion mapscited by 5
  6. 2023Unsupervised Learning on a DIET: Datum IndEx as Target Free of Self-Supervision, Reconstruction, Projector Headcited by 4