# Task-Synchronized Recurrent Neural Networks

arXiv: https://arxiv.org/abs/2204.05192 · HTML: https://mlsanity.com/paper/2204.05192/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 11 Apr 2022.

## Builds on (most cited first, 16 of 16)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Modeling Irregular Time Series with Continuous Recurrent Units](https://mlsanity.com/paper/2111.11344/) (2021)
- [Learning to Diagnose with LSTM Recurrent Neural Networks](https://mlsanity.com/paper/1511.03677/) (2015)
- [Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences](https://mlsanity.com/paper/1610.09513/) (2016)
- [Interpolation-Prediction Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/1909.07782/) (2019)
- [Can recurrent neural networks warp time?](https://mlsanity.com/paper/1804.11188/) (2018)
- [Learning to Detect Sepsis with a Multitask Gaussian Process RNN Classifier](https://mlsanity.com/paper/1706.04152/) (2017)
- [DeepCare: A Deep Dynamic Memory Model for Predictive Medicine](https://mlsanity.com/paper/1602.00357/) (2016)
- [Discrete Event, Continuous Time RNNs](https://mlsanity.com/paper/1710.04110/) (2017)
- [An Improved Multi-Output Gaussian Process RNN with Real-Time Validation for Early Sepsis Detection](https://mlsanity.com/paper/1708.05894/) (2017)
- [Neural Decomposition of Time-Series Data for Effective Generalization](https://mlsanity.com/paper/1705.09137/) (2017)
- [Efficient implementations of echo state network cross-validation](https://mlsanity.com/paper/2006.11282/) (2020)
- [Efficient Cross-Validation of Echo State Networks](https://mlsanity.com/paper/1908.08450/) (2019)

## Built on by (most cited first, 1 of 1)

- [Long-term Forecasting with TiDE: Time-series Dense Encoder](https://mlsanity.com/paper/2304.08424/) (2023)

## Abstract (arXiv)

Data are often sampled irregularly in time. Dealing with this using Recurrent Neural Networks (RNNs) traditionally involved ignoring the fact, feeding the time differences as additional inputs, or resampling the data. All these methods have their shortcomings. We propose an elegant straightforward alternative approach where instead the RNN is in effect resampled in time to match the time of the data or the task at hand. We use Echo State Network (ESN) and Gated Recurrent Unit (GRU) as the basis for our solution. Such RNNs can be seen as discretizations of continuous-time dynamical systems, which gives a solid theoretical ground to our approach. Our Task-Synchronized ESN (TSESN) and GRU (TSGRU) models allow for a direct model time setting and require no additional training, parameter tuning, or computation (solving differential equations or interpolating data) compared to their regular counterparts, thus retaining their original efficiency. We confirm empirically that our models can effectively compensate for the time-non-uniformity of the data and demonstrate that they compare favorably to data resampling, classical RNN methods, and alternative RNN models proposed to deal with time irregularities on several real-world nonuniform-time datasets. We open-source the code at https://github.com/oshapio/task-synchronized-RNNs .