mlsanityData to 7 Oct 2026

Do Image Classifiers Generalize Across Time?

arXiv 1906.02168 ↗First posted 5 Jun 2019cs.LG cs.CV stat.MLcited by 59

Not on the map. Why?

Abstract (arXiv)

We study the robustness of image classifiers to temporal perturbations derived from videos. As part of this study, we construct two datasets, ImageNet-Vid-Robust and YTBB-Robust , containing a total 57,897 images grouped into 3,139 sets of perceptually similar images. Our datasets were derived from ImageNet-Vid and Youtube-BB respectively and thoroughly re-annotated by human experts for image similarity. We evaluate a diverse array of classifiers pre-trained on ImageNet and show a median classification accuracy drop of 16 and 10 on our two datasets. Additionally, we evaluate three detection models and show that natural perturbations induce both classification as well as localization errors, leading to a median drop in detection mAP of 14 points. Our analysis demonstrates that perturbations occurring naturally in videos pose a substantial and realistic challenge to deploying convolutional neural networks in environments that require both reliable and low-latency predictions

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 13; 2023: 13; 2024: 14; 2025: 3; 2026: 1.

59 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  3. 2021Robust fine-tuning of zero-shot modelscited by 785
  4. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 248
  5. 2021Towards Causal Representation Learningcited by 207
  6. 2023Data-centric Artificial Intelligence: A Surveycited by 202
  7. 2021Partial success in closing the gap between human and machine visioncited by 185
  8. 2021A Fine-Grained Analysis on Distribution Shiftcited by 171
  9. 2021Predicting with Confidence on Unseen Distributionscited by 116
  10. 2022Plex: Towards Reliability using Pretrained Large Model Extensionscited by 102
  11. 2021On Interaction Between Augmentations and Corruptions in Natural Corruption Robustnesscited by 98
  12. 2021AugMax: Adversarial Composition of Random Augmentations for Robust Trainingcited by 89
38 more
  1. 2023The effectiveness of MAE pre-pretraining for billion-scale pretrainingcited by 68
  2. 2022Probable Domain Generalization via Quantile Risk Minimizationcited by 66
  3. 2022Assaying Out-Of-Distribution Generalization in Transfer Learningcited by 53
  4. 2021Optimal Representations for Covariate Shiftcited by 51
  5. 2023Test of Time: Instilling Video-Language Models with a Sense of Timecited by 51
  6. 2022Predicting Out-of-Distribution Error with the Projection Normcited by 50
  7. 2024A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP)cited by 44
  8. 2021Compressive Visual Representationscited by 41
  9. 2024A Sober Look at the Robustness of CLIPs to Spurious Featurescited by 38
  10. 2023Diagnosing Model Performance Under Distribution Shiftcited by 37
  11. 2023Test-Time Training on Video Streamscited by 37
  12. 2021Benchmarking the Robustness of Spatial-Temporal Models Against Corruptionscited by 35
  13. 2021CrossNorm and SelfNorm for Generalization under Distribution Shiftscited by 33
  14. 20213DB: A Framework for Debugging Computer Vision Modelscited by 31
  15. 2023Understanding and Mitigating the Label Noise in Pre-training on Downstream Taskscited by 27
  16. 2022Does Robustness on ImageNet Transfer to Downstream Tasks?cited by 20
  17. 2022Self-supervised video pretraining yields robust and more human-aligned visual representationscited by 20
  18. 2024An Empirical Study Into What Matters for Calibrating Vision-Language Modelscited by 20
  19. 2024Investigating the Benefits of Projection Head for Representation Learningcited by 20
  20. 2022CausalAgents: A Robustness Benchmark for Motion Forecasting using Causal Relationshipscited by 18
  21. 2023Domain Generalisation via Risk Distribution Matchingcited by 14
  22. 2022Progress and limitations of deep networks to recognize objects in unusual posescited by 13
  23. 2024Toward a Holistic Evaluation of Robustness in CLIP Modelscited by 13
  24. 2023A Holistic Assessment of the Reliability of Machine Learning Systemscited by 9
  25. 2022A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunitiescited by 8
  26. 2024Impact of Noisy Supervision in Foundation Model Learningcited by 8
  27. 2024Machine Learning Robustness: A Primercited by 8
  28. 2022A Survey on Visual Transfer Learning using Knowledge Graphscited by 7
  29. 2024Ask Your Distribution Shift if Pre-Training is Right for Youcited by 7
  30. 2023RobustCLEVR: A Benchmark and Framework for Evaluating Robustness in Object-centric Learningcited by 6
  31. 2024Benchmarking Zero-Shot Robustness of Multimodal Foundation Models: A Pilot Studycited by 5
  32. 2024Evaluating Model Performance Under Worst-case Subpopulationscited by 5
  33. 2025The PanAf-FGBG Dataset: Understanding the Impact of Backgrounds in Wildlife Behaviour Recognitioncited by 5
  34. 2023Understanding the Robustness of Multi-modal Contrastive Learning to Distribution Shiftcited by 4
  35. 2023On Robustness in Multimodal Learningcited by 3
  36. 2024Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errorscited by 3
  37. 2024LLM Embeddings Improve Test-time Adaptation to Tabular Y|X-Shiftscited by 3
  38. 2022Perception Over Time: Temporal Dynamics for Robust Image Understandingcited by 2

50 most cited of 59 All 59 (JSON)