mlsanityData to 14 Oct 2026

WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series

arXiv 2203.09978 ↗First posted 18 Mar 2022cs.LG stat.MLcited by 30

Not on the map. Why?

Abstract (arXiv)

Machine learning models often fail to generalize well under distributional shifts. Understanding and overcoming these failures have led to a research field of Out-of-Distribution (OOD) generalization. Despite being extensively studied for static computer vision tasks, OOD generalization has been underexplored for time series tasks. To shine light on this gap, we present WOODS: eight challenging open-source time series benchmarks covering a diverse range of data modalities, such as videos, brain recordings, and sensor signals. We revise the existing OOD generalization algorithms for time series tasks and evaluate them using our systematic framework. Our experiments show a large room for improvement for empirical risk minimization and OOD generalization algorithms on our datasets, thus underscoring the new challenges posed by time series tasks. Code and documentation are available at https://woods-benchmarks.github.io .

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 90 of its 167 references that are in the map. First posted on arXiv 18 Mar 2022.

Builds on

90 of 167 references.

References · 90

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  9. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  10. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  11. 2019Invariant Risk Minimizationcited by 1,854
  12. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
38 more
  1. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  2. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  3. 2019Natural Adversarial Examplescited by 1,408
  4. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  5. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  6. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
  7. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  8. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,200
  9. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,188
  10. 2020In Search of Lost Domain Generalizationcited by 1,054
  11. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,013
  12. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
  13. 2017Deeper, Broader and Artier Domain Generalizationcited by 986
  14. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  15. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 742
  16. 2015Image-based Recommendations on Styles and Substitutescited by 706
  17. 2018Recognition in Terra Incognitacited by 658
  18. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
  19. 2016EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfacescited by 628
  20. 2018Deep learning for time series classification: a reviewcited by 576
  21. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 536
  22. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 523
  23. 2016Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filteringcited by 520
  24. 2020Time Series Forecasting With Deep Learning: A Surveycited by 459
  25. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 436
  26. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 343
  27. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 335
  28. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 294
  29. 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 284
  30. 2018Scalable and accurate deep learning for electronic health recordscited by 280
  31. 2019Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019cited by 273
  32. 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 267
  33. 2019Compositionality decomposed: how do neural networks generalise?cited by 265
  34. 2020COGS: A Compositional Generalization Challenge Based on Semantic Interpretationcited by 263
  35. 2015Domain Generalization for Object Recognition with Multi-task Autoencoderscited by 253
  36. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 249
  37. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 230
  38. 2020Time Series Data Augmentation for Deep Learning: A Surveycited by 226

50 most cited of 90 All 90 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 11; 2024: 7; 2025: 6; 2026: 4.

30 papers.

Sort:

  1. 2022GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspectivecited by 72
  2. 2023Free Lunch for Domain Adversarial Training: Environment Label Smoothingcited by 33
  3. 2022GLOBEM Dataset: Multi-Year Datasets for Longitudinal Human Behavior Modeling Generalizationcited by 28
  4. 2023TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Seriescited by 26
  5. 2023Environment-Aware Dynamic Graph Learning for Out-of-Distribution Generalizationcited by 26
  6. 2025Mantis: Lightweight Foundation Model for Time Series Classificationcited by 24
  7. 2024Spectral Invariant Learning for Dynamic Graphs under Distribution Shiftscited by 19
  8. 2024Domain Generalisation via Imprecise Learningcited by 14
  9. 2023Domain Generalization in Machine Learning Models for Wireless Communications: Concepts, State-of-the-Art, and Open Issuescited by 12
  10. 2026MantisV2: Closing the Zero-Shot Gap in Time Series Classification with Synthetic Data and Test-Time Strategiescited by 11
  11. 2023Out-of-Distribution Generalized Dynamic Graph Neural Network with Disentangled Intervention and Invariance Promotioncited by 8
  12. 2023Wild-Tab: A Benchmark For Out-Of-Distribution Generalization In Tabular Regressioncited by 6
18 more
  1. 2024Enhancing Evolving Domain Generalization through Dynamic Latent Representationscited by 6
  2. 2025Unlocking Pretrained Vision Transformers for Time Series Classificationcited by 6
  3. 2023An Empirical Study of Pre-trained Model Selection for Out-of-Distribution Generalization and Calibrationcited by 5
  4. 2023Out-of-Distribution Generalized Dynamic Graph Neural Network for Human Albumin Predictioncited by 5
  5. 2024Phase-driven Domain Generalizable Learning for Nonstationary Time Seriescited by 5
  6. 2023RELand: Risk Estimation of Landmines via Interpretable Invariant Risk Minimizationcited by 3
  7. 2024SKADA-Bench: Benchmarking Unsupervised Domain Adaptation Methods with Realistic Validation On Diverse Modalitiescited by 2
  8. 2024Con4m: Context-aware Consistency Learning Framework for Segmented Time Series Classificationcited by 2
  9. 2025Towards Generalizable PDE Dynamics Forecasting via Physics-Guided Invariant Learningcited by 2
  10. 2025Evolving Graph Learning for Out-of-Distribution Generalization in Non-stationary Environmentscited by 2
  11. 2023Out-of-Distribution Generalization in Text Classification: Past, Present, and Futurecited by 1
  12. 2023Context is Environmentcited by 1
  13. 2025Chorus: Harmonizing Context and Sensing Signals for Data-Free Model Customization in IoTcited by 1
  14. 2026Assessing the Robustness of Climate Foundation Models under No-Analog Distribution Shiftscited by 1
  15. 2024TADA: Temporal Adversarial Data Augmentation for Time Series Datacited by 0
  16. 2025Do machine learning climate models work in changing climate dynamics?cited by 0
  17. 2026TS-Fault: Benchmarking Time Series Forecasters Against Structural Faultscited by 0
  18. 2026Time Series Forecasting Benchmarks Need Scenario-Grounded Stress Testingcited by 0