mlsanityData to 7 Oct 2026

Attentive Neural Controlled Differential Equations for Time-series Classification and Forecasting

arXiv 2109.01876 ↗First posted 4 Sep 2021cs.LG stat.MLcited by 28

Not on the map. Why?

Abstract (arXiv)

Neural networks inspired by differential equations have proliferated for the past several years. Neural ordinary differential equations (NODEs) and neural controlled differential equations (NCDEs) are two representative examples of them. In theory, NCDEs provide better representation learning capability for time-series data than NODEs. In particular, it is known that NCDEs are suitable for processing irregular time-series data. Whereas NODEs have been successfully extended after adopting attention, however, it had not been studied yet how to integrate attention into NCDEs. To this end, we present the method of Attentive Neural Controlled Differential Equations (ANCDEs) for time-series classification and forecasting, where dual NCDEs are used: one for generating attention values, and the other for evolving hidden vectors for a downstream machine learning task. We conduct experiments with three real-world time-series datasets and 10 baselines. After dropping some values, we also conduct irregular time-series experiments. Our method consistently shows the best accuracy in all cases by non-trivial margins. Our visualizations also show that the presented attention mechanism works as intended by focusing on crucial information.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 31 of its 80 references that are in the map. First posted on arXiv 4 Sep 2021.

Builds on

31 of 80 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  5. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  6. 2018DeepMind Control Suitecited by 935
  7. 2018Deep learning for time series classification: a reviewcited by 575
  8. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 548
  9. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  10. 2019Augmented Neural ODEscited by 396
  11. 2018The UEA multivariate time series classification archive, 2018cited by 330
  12. 2019Attentive Neural Processescited by 280
19 more
  1. 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278
  2. 2016Image Captioning with Semantic Attentioncited by 260
  3. 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 230
  4. 2014Recurrent Neural Network Regularizationcited by 189
  5. 2019Latent ODEs for Irregularly-Sampled Time Seriescited by 182
  6. 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 168
  7. 2019An Attentive Survey of Attention Modelscited by 123
  8. 2019Attention in Natural Language Processingcited by 100
  9. 2020Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODEcited by 76
  10. 2017DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understandingcited by 71
  11. 2021MALI: A memory efficient and reverse accurate integrator for Neural ODEscited by 42
  12. 2018Attention Models in Graphs: A Surveycited by 41
  13. 2019Neural Dynamics on Complex Networkscited by 38
  14. 2020Neural Jump Ordinary Differential Equations: Consistent Continuous-Time Prediction and Filteringcited by 33
  15. 2015Describing Multimedia Content using Attention-based Encoder--Decoder Networkscited by 32
  16. 2019Gated recurrent units viewed through the lens of continuous time dynamical systemscited by 18
  17. 2018Dynamic Meta-Embeddings for Improved Sentence Representationscited by 15
  18. 2021ACE-NODE: Attentive Co-Evolving Neural Ordinary Differential Equationscited by 15
  19. 2020Explainable Tensorized Neural Ordinary Differential Equations forArbitrary-step Time Series Predictioncited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 3; 2024: 8; 2025: 7; 2026: 6.

28 papers.

Sort:

By area:

  1. 2024ContiFormer: Continuous-Time Transformer for Irregular Time Series Modelingcited by 61
  2. 2022GT-GAN: General Purpose Time Series Synthesis with Generative Adversarial Networkscited by 43
  3. 2024Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Datacited by 35
  4. 2024Universal Time-Series Representation Learning: A Surveycited by 28
  5. 2022EXIT: Extrapolation and Interpolation-based Neural Controlled Differential Equations for Time-series Classification and Forecastingcited by 22
  6. 2025Comprehensive Review of Neural Differential Equations for Time Series Analysiscited by 20
  7. 2023Learnable Path in Neural Controlled Differential Equationscited by 8
  8. 2023Signed Graph Neural Ordinary Differential Equation for Modeling Continuous-time Dynamicscited by 8
  9. 2022TimeKit: A Time-series Forecasting-based Upgrade Kit for Collaborative Filteringcited by 7
  10. 2024Addressing Prediction Delays in Time Series Forecasting: A Continuous GRU Approach with Derivative Regularizationcited by 7
  11. 2025HD-NDEs: Neural Differential Equations for Hallucination Detection in LLMscited by 6
  12. 2026Tracing Agentic Failure from the Flow of Successcited by 5
16 more
  1. 2024DualDynamics: Synergizing Implicit and Explicit Methods for Robust Irregular Time Series Analysiscited by 4
  2. 2025Neural Predictive Control to Coordinate Discrete- and Continuous-Time Models for Time-Series Analysis with Control-Theoretical Improvementscited by 3
  3. 2022LORD: Lower-Dimensional Embedding of Log-Signature in Neural Rough Differential Equationscited by 2
  4. 2024GDFlow: Anomaly Detection with NCDE-based Normalizing Flow for Advanced Driver Assistance Systemcited by 2
  5. 2025FlowPath: Learning Data-Driven Manifolds with Invertible Flows for Robust Irregularly-sampled Time Series Classificationcited by 2
  6. 2023Long-term Time Series Forecasting based on Decomposition and Neural Ordinary Differential Equationscited by 1
  7. 2024Bridging Dynamic Factor Models and Neural Controlled Differential Equations for Nowcasting GDPcited by 1
  8. 2025Continuum Dropout for Neural Differential Equationscited by 1
  9. 2026Efficient Neural Controlled Differential Equations via Attentive Kernel Smoothingcited by 1
  10. 2024Spatial Shortcuts in Graph Neural Controlled Differential Equationscited by 0
  11. 2025TANDEM: Temporal Attention-guided Neural Differential Equations for Missingness in Time Series Classificationcited by 0
  12. 2025Random Controlled Differential Equationscited by 0
  13. 2026U-Former ODE: Fast Probabilistic Forecasting of Irregular Time Seriescited by 0
  14. 2026INDEQS: Informed Neural controlled Differential EQuationScited by 0
  15. 2026Continuous-Time Acoustic Modelling with Neural Controlled Differential Equationscited by 0
  16. 2026Continuous-Time Machine Learning: A Unified Mathematical Perspectivecited by 0