# Out-of-Distribution Representation Learning for Time Series Classification

arXiv: https://arxiv.org/abs/2209.07027 · HTML: https://mlsanity.com/paper/2209.07027/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 67 references that are in the map.
- First posted on arXiv 15 Sep 2022.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Self-Challenging Improves Cross-Domain Generalization](https://mlsanity.com/paper/2007.02454/) (2020)
- [MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classification](https://mlsanity.com/paper/2012.08791/) (2020)
- [Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization](https://mlsanity.com/paper/2109.02934/) (2021)
- [Domain Generalization Using a Mixture of Multiple Latent Domains](https://mlsanity.com/paper/1911.07661/) (2019)
- [Learning to Diversify for Single Domain Generalization](https://mlsanity.com/paper/2108.11726/) (2021)
- [Deep Learning for Sensor-based Activity Recognition: A Survey](https://mlsanity.com/paper/1707.03502/) (2017)
- [Learning explanations that are hard to vary](https://mlsanity.com/paper/2009.00329/) (2020)
- [AdaRNN: Adaptive Learning and Forecasting of Time Series](https://mlsanity.com/paper/2108.04443/) (2021)
- [Domain Agnostic Learning with Disentangled Representations](https://mlsanity.com/paper/1904.12347/) (2019)
- [Progressive Domain Expansion Network for Single Domain Generalization](https://mlsanity.com/paper/2103.16050/) (2021)
- [Towards Principled Disentanglement for Domain Generalization](https://mlsanity.com/paper/2111.13839/) (2021)
- [Adversarially Adaptive Normalization for Single Domain Generalization](https://mlsanity.com/paper/2106.01899/) (2021)
- [Delving Deep into the Generalization of Vision Transformers under Distribution Shifts](https://mlsanity.com/paper/2106.07617/) (2021)
- [Multi-Source Deep Domain Adaptation with Weak Supervision for Time-Series Sensor Data](https://mlsanity.com/paper/2005.10996/) (2020)

## Built on by (most cited first, 30 of 31)

- [UniTS: A Unified Multi-Task Time Series Model](https://mlsanity.com/paper/2403.00131/) (2024)
- [SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalization](https://mlsanity.com/paper/2310.19795/) (2023)
- [Time-Series Forecasting for Out-of-Distribution Generalization Using Invariant Learning](https://mlsanity.com/paper/2406.09130/) (2024)
- [Towards Multimodal Open-Set Domain Generalization and Adaptation through Self-supervision](https://mlsanity.com/paper/2407.01518/) (2024)
- [Out-of-Distribution Generalization in Time Series: A Survey](https://mlsanity.com/paper/2503.13868/) (2025)
- [SciTS: Scientific Time Series Understanding and Generation with LLMs](https://mlsanity.com/paper/2510.03255/) (2025)
- [Unsupervised Representation Learning for Time Series: A Review](https://mlsanity.com/paper/2308.01578/) (2023)
- [Tri-Level Navigator: LLM-Empowered Tri-Level Learning for Time Series OOD Generalization](https://mlsanity.com/paper/2410.07018/) (2024)
- [Evaluating the Generalization Ability of Spatiotemporal Model in Urban Scenario](https://mlsanity.com/paper/2410.04740/) (2024)
- [Continuous Invariance Learning](https://mlsanity.com/paper/2310.05348/) (2023)
- [Simulations of Common Unsupervised Domain Adaptation Algorithms for Image Classification](https://mlsanity.com/paper/2502.10694/) (2025)
- [StruSR: Structure-Aware Symbolic Regression with Physics-Informed Taylor Guidance](https://mlsanity.com/paper/2510.06635/) (2025)
- [Phase-driven Domain Generalizable Learning for Nonstationary Time Series](https://mlsanity.com/paper/2402.05960/) (2024)
- [Target-agnostic Source-free Domain Adaptation for Regression Tasks](https://mlsanity.com/paper/2312.00540/) (2023)
- [Deep Generative Domain Adaptation with Temporal Attention for Cross-User Activity Recognition](https://mlsanity.com/paper/2403.17958/) (2024)
- [A Scene-aware Models Adaptation Scheme for Cross-scene Online Inference on Mobile Devices](https://mlsanity.com/paper/2407.03331/) (2024)
- [FredNormer: Frequency Domain Normalization for Non-stationary Time Series Forecasting](https://mlsanity.com/paper/2410.01860/) (2024)
- [HAR-DoReMi: Optimizing Data Mixture for Self-Supervised Human Activity Recognition Across Heterogeneous IMU Datasets](https://mlsanity.com/paper/2503.13542/) (2025)
- [Continuous Domain Generalization](https://mlsanity.com/paper/2505.13519/) (2025)
- [SSAS: Cross-subject EEG-based Emotion Recognition through Source Selection with Adversarial Strategy](https://mlsanity.com/paper/2512.13458/) (2025)
- [Prism: Mining Task-aware Domains in Non-i.i.d. IMU Data for Flexible User Perception](https://mlsanity.com/paper/2501.01598/) (2025)
- [Tackling Time-Series Forecasting Generalization via Mitigating Concept Drift](https://mlsanity.com/paper/2510.14814/) (2025)
- [Cross-user activity recognition using deep domain adaptation with temporal relation information](https://mlsanity.com/paper/2403.15424/) (2024)
- [Elevator, Escalator, or Neither? Classifying Conveyor State Using Smartphone under Arbitrary Pedestrian Behavior](https://mlsanity.com/paper/2405.03218/) (2024)
- [Adversarial Domain Adaptation for Cross-user Activity Recognition Using Diffusion-based Noise-centred Learning](https://mlsanity.com/paper/2408.03353/) (2024)
- [Learning Latent Spaces for Domain Generalization in Time Series Forecasting](https://mlsanity.com/paper/2412.11171/) (2024)
- [ERIS: An Energy-Guided Feature Disentanglement Framework for Out-of-Distribution Time Series Classification](https://mlsanity.com/paper/2508.14134/) (2025)
- [Out of Distribution Detection for Efficient Continual Learning in Quality Prediction for Arc Welding](https://mlsanity.com/paper/2508.16832/) (2025)
- [ADAPTOOD: Uncertainty-Aware Fine-Tuning for Out-of-Distribution ECG Time Series Models](https://mlsanity.com/paper/2606.04164/) (2026)
- [Domain Generalization under Sampling Pattern Shifts in Irregular Time Series](https://mlsanity.com/paper/2609.33279/) (2026)

## Abstract (arXiv)

Time series classification is an important problem in real world. Due to its non-stationary property that the distribution changes over time, it remains challenging to build models for generalization to unseen distributions. In this paper, we propose to view the time series classification problem from the distribution perspective. We argue that the temporal complexity attributes to the unknown latent distributions within. To this end, we propose DIVERSIFY to learn generalized representations for time series classification. DIVERSIFY takes an iterative process: it first obtains the worst-case distribution scenario via adversarial training, then matches the distributions of the obtained sub-domains. We also present some theoretical insights. We conduct experiments on gesture recognition, speech commands recognition, wearable stress and affect detection, and sensor-based human activity recognition with a total of seven datasets in different settings. Results demonstrate that DIVERSIFY significantly outperforms other baselines and effectively characterizes the latent distributions by qualitative and quantitative analysis. Code is available at: https://github.com/microsoft/robustlearn.