mlsanityData to 14 Oct 2026

Learning to See by Looking at Noise

arXiv 2106.05963 ↗First posted 10 Jun 2021cs.CV cs.AIcited by 92

Not on the map. Why?

Abstract (arXiv)

Current vision systems are trained on huge datasets, and these datasets come with costs: curation is expensive, they inherit human biases, and there are concerns over privacy and usage rights. To counter these costs, interest has surged in learning from cheaper data sources, such as unlabeled images. In this paper we go a step further and ask if we can do away with real image datasets entirely, instead learning from noise processes. We investigate a suite of image generation models that produce images from simple random processes. These are then used as training data for a visual representation learner with a contrastive loss. We study two types of noise processes, statistical image models and deep generative models under different random initializations. Our findings show that it is important for the noise to capture certain structural properties of real data but that good performance can be achieved even with processes that are far from realistic. We also find that diversity is a key property to learn good representations. Datasets, models, and code are available at https://mbaradad.github.io/learning_with_noise.

Key facts. Cited by 92 papers in the map (data through 14 Oct 2026). Builds on 33 of its 68 references that are in the map. First posted on arXiv 10 Jun 2021.

Builds on

33 of 68 references.

References · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  7. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  8. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  9. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  10. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  11. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  12. 2019Contrastive Multiview Codingcited by 1,301
21 more
  1. 2017Deep Image Priorcited by 1,098
  2. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  3. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  4. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
  5. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  6. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  7. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 425
  8. 2019SinGAN: Learning a Generative Model from a Single Natural Imagecited by 348
  9. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  10. 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
  11. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 215
  12. 2021Self-supervised Pretraining of Visual Features in the Wildcited by 198
  13. 2021Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalizationcited by 141
  14. 2019Classification Accuracy Score for Conditional Generative Modelscited by 136
  15. 2021Generative Models as a Data Source for Multiview Representation Learningcited by 104
  16. 2021Pre-training without Natural Imagescited by 100
  17. 2021Repurposing GANs for One-shot Semantic Part Segmentationcited by 88
  18. 2019A critical analysis of self-supervision, or what we can learn from a single imagecited by 76
  19. 2019This dataset does not exist: training models from generated imagescited by 65
  20. 2018What Makes Good Synthetic Training Data for Learning Disparity and Optical Flow Estimation?cited by 44
  21. 2020Harmonic Convolutional Networks based on Discrete Cosine Transformcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 14; 2023: 15; 2024: 24; 2025: 16; 2026: 18.

92 papers.

Sort:

By area:

  1. 2024The Platonic Representation Hypothesiscited by 382
  2. 2023StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learnerscited by 177
  3. 2021PixMix: Dreamlike Pictures Comprehensively Improve Safety Measurescited by 131
  4. 2023Generalizing Dataset Distillation via Deep Generative Priorcited by 126
  5. 2023Scaling Laws of Synthetic Images for Model Training ... for Nowcited by 108
  6. 2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
  7. 2021GAN-Supervised Dense Visual Alignmentcited by 62
  8. 2023FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Modelscited by 61
  9. 2022Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learningcited by 52
  10. 2024A Vision Check-up for Language Modelscited by 36
  11. 2022ModelDiff: A Framework for Comparing Learning Algorithmscited by 30
  12. 2024Learning General-Purpose Biomedical Volume Representations using Randomized Synthesiscited by 29
38 more
  1. 2022Procedural Image Programs for Representation Learningcited by 28
  2. 2021Task2Sim : Towards Effective Pre-training and Transfer from Synthetic Datacited by 26
  3. 2024Do Generated Data Always Help Contrastive Learning?cited by 25
  4. 2022Pre-training Vision Transformers with Formula-driven Supervised Learningcited by 21
  5. 2023Differentially Private Image Classification by Learning Priors from Random Processescited by 19
  6. 2024The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Bettercited by 19
  7. 2024Deep Linear Probe Generators for Weight Space Learningcited by 17
  8. 2024FreeTumor: Advance Tumor Segmentation via Large-Scale Tumor Synthesiscited by 15
  9. 2026Training Language Models via Neural Cellular Automatacited by 15
  10. 2023IPMix: Label-Preserving Data Augmentation Method for Training Robust Classifierscited by 14
  11. 2024Scaling Backwards: Minimal Synthetic Pre-training?cited by 14
  12. 2025General Intelligence Requires Reward-based Pretrainingcited by 12
  13. 2025Enhancing Vision-Language Compositional Understanding with Multimodal Synthetic Datacited by 12
  14. 2023DMLR: Data-centric Machine Learning Research -- Past, Present and Futurecited by 11
  15. 2026Procedural Pretraining: Warming Up Language Models with Abstract Datacited by 10
  16. 2024What Makes a Good Dataset for Knowledge Distillation?cited by 9
  17. 2025Transformers Pretrained on Procedural Data Contain Modular Structures for Algorithmic Reasoningcited by 9
  18. 2022Visual Pre-training for Navigation: What Can We Learn from Noise?cited by 8
  19. 2025Can You Learn to See Without Images? Procedural Warm-Up for Vision Transformerscited by 8
  20. 2021The Augmented Image Prior: Distilling 1000 Classes by Extrapolating from a Single Imagecited by 7
  21. 2022Palm up: Playing in the Latent Manifold for Unsupervised Pretrainingcited by 7
  22. 2022Adjusting for Bias with Procedural Datacited by 6
  23. 2022Commonality in Natural Images Rescues GANs: Pretraining GANs with Generic and Privacy-free Synthetic Datacited by 6
  24. 2026When Pretty Isn't Useful: Investigating Why Modern Text-to-Image Models Fail as Reliable Training Data Generatorscited by 6
  25. 2021Compositional generalization in semantic parsing with pretrained transformerscited by 5
  26. 2023Learning to Count without Annotationscited by 5
  27. 2023Model Synthesis for Zero-Shot Model Attributioncited by 5
  28. 2022Extreme Image Transformations Affect Humans and Machines Differentlycited by 4
  29. 2023Projected Latent Distillation for Data-Agnostic Consolidation in Distributed Continual Learningcited by 4
  30. 2023USIM-DAL: Uncertainty-aware Statistical Image Modeling-based Dense Active Learning for Super-resolutioncited by 4
  31. 2023Harnessing Synthetic Datasets: The Role of Shape Bias in Deep Neural Network Generalizationcited by 4
  32. 2024Communication-Efficient Federated Learning through Adaptive Weight Clustering and Server-Side Distillationcited by 4
  33. 2024Characterizing the Training Dynamics of Private Fine-tuning with Langevin diffusioncited by 4
  34. 2024Efficient Diffusion-Driven Corruption Editor for Test-Time Adaptationcited by 4
  35. 2022Robustness of Humans and Machines on Object Recognition with Extreme Image Transformationscited by 3
  36. 2023Hate Speech Detection in Limited Data Contexts using Synthetic Data Generationcited by 3
  37. 2024Can Generative Models Improve Self-Supervised Representation Learning?cited by 3
  38. 2024Pre-training with Synthetic Patterns for Audiocited by 3

50 most cited of 92 All 92 (JSON)