mlsanityData to 7 Oct 2026

MixDiff: Mixing Natural and Synthetic Images for Robust Self-Supervised Representations

arXiv 2406.12368 ↗First posted 18 Jun 2024cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

This paper introduces MixDiff, a new self-supervised learning (SSL) pre-training framework that combines real and synthetic images. Unlike traditional SSL methods that predominantly use real images, MixDiff uses a variant of Stable Diffusion to replace an augmented instance of a real image, facilitating the learning of cross real-synthetic image representations. Our key insight is that while models trained solely on synthetic images underperform, combining real and synthetic data leads to more robust and adaptable representations. Experiments show MixDiff enhances SimCLR, BarlowTwins, and DINO across various robustness datasets and domain transfer tasks, boosting SimCLR's ImageNet-1K accuracy by 4.56%. Our framework also demonstrates comparable performance without needing any augmentations, a surprising finding in SSL where augmentations are typically crucial.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 67 of its 85 references that are in the map. First posted on arXiv 18 Jun 2024.

Builds on

67 of 85 references.

References · 67

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  9. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  12. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
38 more
  1. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  3. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  5. 2013Describing Textures in the Wildcited by 2,110
  6. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  7. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  8. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  9. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  10. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  11. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  12. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  13. 2019Natural Adversarial Examplescited by 1,401
  14. 2019Contrastive Multiview Codingcited by 1,297
  15. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  16. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  17. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  18. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  19. 2017Large Batch Training of Convolutional Networkscited by 529
  20. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
  21. 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 326
  22. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  23. 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 299
  24. 2018Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomizationcited by 292
  25. 2023Effective Data Augmentation With Diffusion Modelscited by 282
  26. 2019Unsupervised Embedding Learning via Invariant and Spreading Instance Featurecited by 231
  27. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
  28. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  29. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  30. 2022Versatile Diffusion: Text, Images and Variations All in One Diffusion Modelcited by 199
  31. 2023StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learnerscited by 177
  32. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 171
  33. 2020Knowledge Distillation Meets Self-Supervisioncited by 142
  34. 2018Learning Semantic Segmentation from Synthetic Data: A Geometrically Guided Input-Output Adaptation Approachcited by 119
  35. 2021Can contrastive learning avoid shortcut solutions?cited by 114
  36. 2023Scaling Laws of Synthetic Images for Model Training ... for Nowcited by 107
  37. 2021AutoFlow: Learning a Better Training Set for Optical Flowcited by 102
  38. 2023Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentationcited by 100

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Improving watermelon (Citrullus lanatus) disease classification with generative artificial intelligence (GenAI)-based synthetic and real-field images via a custom EfficientNetV2-L modelcited by 0
  2. 2026Chaotic Contrastive Learning for Robust Texture Classificationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Is Synthetic Data all We Need? Benchmarking the Robustness of Models Trained with Synthetic Images
  2. 2024Contrastive Learning with Synthetic Positives
  3. 2023Real-Fake: Effective Training Data Synthesis Through Distribution Matching
  4. 2023Diversity and Diffusion: Observations on Synthetic Image Distributions with Stable Diffusion
  5. 2024Mind the Gap Between Synthetic and Real: Utilizing Transfer Learning to Probe the Boundaries of Stable Diffusion Generated Data
  6. 2024The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better
  7. 2023On the Stability of Iterative Retraining of Generative Models on their own Data
  8. 2023DP-Mix: Mixup-based Data Augmentation for Differentially Private Learning