mlsanityData to 7 Oct 2026

The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization

arXiv 2006.16241 ↗First posted 29 Jun 2020cs.CV cs.LG stat.MLcited by 1,842

Not on the map. Why?

Cited by 367 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce four new real-world distribution shift datasets consisting of changes in image style, image blurriness, geographic location, camera operation, and more. With our new datasets, we take stock of previously proposed methods for improving out-of-distribution robustness and put them to the test. We find that using larger models and artificial data augmentations can improve robustness on real-world distribution shifts, contrary to claims in prior work. We find improvements in artificial robustness benchmarks can transfer to real-world distribution shifts, contrary to claims in prior work. Motivated by our observation that data augmentations can help with real-world distribution shifts, we also introduce a new data augmentation method which advances the state-of-the-art and outperforms models pretrained with 1000 times more labeled data. Overall we find that some methods consistently help with distribution shifts in texture and local image statistics, but these methods do not help with some other distribution shifts like geographic changes. Our results show that future research must study multiple distribution shifts simultaneously, as we demonstrate that no evaluated method consistently improves robustness.

Key facts. Cited by 1,842 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 137; 2022: 262; 2023: 359; 2024: 444; 2025: 380; 2026: 260.

1,842 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  5. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  6. 2022A ConvNet for the 2020scited by 3,545
  7. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  8. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  9. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  10. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  11. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  12. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
38 more
  1. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  2. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  3. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  4. 2021Robust fine-tuning of zero-shot modelscited by 785
  5. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  6. 2021Domain Generalization: A Surveycited by 760
  7. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  8. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  9. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  10. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  11. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
  12. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  13. 2021Merging Models with Fisher-Weighted Averagingcited by 613
  14. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  15. 2022Continual Test-Time Domain Adaptationcited by 520
  16. 2022Efficient Test-Time Model Adaptation without Forgettingcited by 467
  17. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
  18. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  19. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  20. 2023Symbolic Discovery of Optimization Algorithmscited by 423
  21. 2023Towards Stable Test-Time Adaptation in Dynamic Wild Worldcited by 409
  22. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  23. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  24. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Modelscited by 379
  25. 2023EVA-02: A Visual Representation for Neon Genesiscited by 377
  26. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 376
  27. 2022Prompt-aligned Gradient for Prompt Tuningcited by 339
  28. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
  29. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  30. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 315
  31. 2023Visual-Language Prompt Tuning with Knowledge-guided Context Optimizationcited by 307
  32. 2022Masked Siamese Networks for Label-Efficient Learningcited by 301
  33. 2021DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentationcited by 299
  34. 2022BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizerscited by 298
  35. 2023SemDeDup: Data-efficient learning at web-scale through semantic deduplicationcited by 295
  36. 2022Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuningcited by 292
  37. 2021Unsolved Problems in ML Safetycited by 291
  38. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 287

50 most cited of 1,842 All 1,842 (JSON)