# Continual Learning for Multivariate Time Series Tasks with Variable Input Dimensions

arXiv: https://arxiv.org/abs/2203.06852 · HTML: https://mlsanity.com/paper/2203.06852/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 14 Mar 2022.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Neural Module Networks](https://mlsanity.com/paper/1511.02799/) (2015)
- [CondConv: Conditionally Parameterized Convolutions for Efficient Inference](https://mlsanity.com/paper/1904.04971/) (2019)
- [Selective Experience Replay for Lifelong Learning](https://mlsanity.com/paper/1802.10269/) (2018)
- [Supermasks in Superposition](https://mlsanity.com/paper/2006.14769/) (2020)
- [Universal Source-Free Domain Adaptation](https://mlsanity.com/paper/2004.04393/) (2020)
- [Deep Learning for Sensor-based Activity Recognition: A Survey](https://mlsanity.com/paper/1707.03502/) (2017)
- [Multivariate LSTM-FCNs for Time Series Classification](https://mlsanity.com/paper/1801.04503/) (2018)
- [Routing Networks and the Challenges of Modular and Compositional Computation](https://mlsanity.com/paper/1904.12774/) (2019)
- [TimeNet: Pre-trained deep recurrent neural network for time series classification](https://mlsanity.com/paper/1706.08838/) (2017)
- [Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies](https://mlsanity.com/paper/1808.06508/) (2018)
- [Data augmentation using synthetic data for time series classification with deep residual networks](https://mlsanity.com/paper/1808.02455/) (2018)
- [Modular Universal Reparameterization: Deep Multi-task Learning Across Diverse Domains](https://mlsanity.com/paper/1906.00097/) (2019)
- [Graph Neural Networks for Leveraging Industrial Equipment Structure: An application to Remaining Useful Life Estimation](https://mlsanity.com/paper/2006.16556/) (2020)
- [Handling Variable-Dimensional Time Series with Graph Neural Networks](https://mlsanity.com/paper/2007.00411/) (2020)

## Built on by (most cited first, 14 of 14)

- [Design Principles for Lifelong Learning AI Accelerators](https://mlsanity.com/paper/2310.04467/) (2023)
- [Class-incremental Learning for Time Series: Benchmark and Evaluation](https://mlsanity.com/paper/2402.12035/) (2024)
- [Dataset Condensation for Time Series Classification via Dual Domain Matching](https://mlsanity.com/paper/2403.07245/) (2024)
- [Continual Learning for Smart City: A Survey](https://mlsanity.com/paper/2404.00983/) (2024)
- [Structural Knowledge Informed Continual Multivariate Time Series Forecasting](https://mlsanity.com/paper/2402.12722/) (2024)
- [A Natural Gas Consumption Forecasting System for Continual Learning Scenarios based on Hoeffding Trees with Change Point Detection Mechanism](https://mlsanity.com/paper/2309.03720/) (2023)
- [DynaConF: Dynamic Forecasting of Non-Stationary Time Series](https://mlsanity.com/paper/2209.08411/) (2022)
- [Self-supervised Activity Representation Learning with Incremental Data: An Empirical Study](https://mlsanity.com/paper/2305.00619/) (2023)
- [One-Shot Domain Incremental Learning](https://mlsanity.com/paper/2403.16707/) (2024)
- [Predicting Temporal Aspects of Movement for Predictive Replication in Fog Environments](https://mlsanity.com/paper/2306.00575/) (2023)
- [ODEStream: A Buffer-Free Online Learning Framework with ODE-based Adaptor for Streaming Time Series Forecasting](https://mlsanity.com/paper/2411.07413/) (2024)
- [Temporal Taskification in Streaming Continual Learning: A Source of Evaluation Instability](https://mlsanity.com/paper/2604.21930/) (2026)
- [Challenges of Explainability in Continual Learning for Time Series Forecasting](https://mlsanity.com/paper/2607.19382/) (2026)
- [Attention-based Experience Replay Framework for Continual Learning of Agnostic Time Series Forecasting Models](https://mlsanity.com/paper/2607.20493/) (2026)

## Abstract (arXiv)

We consider a sequence of related multivariate time series learning tasks, such as predicting failures for different instances of a machine from time series of multi-sensor data, or activity recognition tasks over different individuals from multiple wearable sensors. We focus on two under-explored practical challenges arising in such settings: (i) Each task may have a different subset of sensors, i.e., providing different partial observations of the underlying 'system'. This restriction can be due to different manufacturers in the former case, and people wearing more or less measurement devices in the latter (ii) We are not allowed to store or re-access data from a task once it has been observed at the task level. This may be due to privacy considerations in the case of people, or legal restrictions placed by machine owners. Nevertheless, we would like to (a) improve performance on subsequent tasks using experience from completed tasks as well as (b) continue to perform better on past tasks, e.g., update the model and improve predictions on even the first machine after learning from subsequently observed ones. We note that existing continual learning methods do not take into account variability in input dimensions arising due to different subsets of sensors being available across tasks, and struggle to adapt to such variable input dimensions (VID) tasks. In this work, we address this shortcoming of existing methods. To this end, we learn task-specific generative models and classifiers, and use these to augment data for target tasks. Since the input dimensions across tasks vary, we propose a novel conditioning module based on graph neural networks to aid a standard recurrent neural network. We evaluate the efficacy of the proposed approach on three publicly available datasets corresponding to two activity recognition tasks (classification) and one prognostics task (regression).