mlsanityData to 7 Oct 2026

ACE-NODE: Attentive Co-Evolving Neural Ordinary Differential Equations

arXiv 2105.14953 ↗First posted 31 May 2021cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Neural ordinary differential equations (NODEs) presented a new paradigm to construct (continuous-time) neural networks. While showing several good characteristics in terms of the number of parameters and the flexibility in constructing neural networks, they also have a couple of well-known limitations: i) theoretically NODEs learn homeomorphic mapping functions only, and ii) sometimes NODEs show numerical instability in solving integral problems. To handle this, many enhancements have been proposed. To our knowledge, however, integrating attention into NODEs has been overlooked for a while. To this end, we present a novel method of attentive dual co-evolving NODE (ACE-NODE): one main NODE for a downstream machine learning task and the other for providing attention to the main NODE. Our ACE-NODE supports both pairwise and elementwise attention. In our experiments, our method outperforms existing NODE-based and non-NODE-based baselines in almost all cases by non-trivial margins.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 27 of its 46 references that are in the map. First posted on arXiv 31 May 2021.

Builds on

27 of 46 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  5. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  6. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  7. 2014Neural Turing Machinescited by 630
  8. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 548
  9. 2019Augmented Neural ODEscited by 396
  10. 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278
  11. 2016Image Captioning with Semantic Attentioncited by 260
  12. 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 230
15 more
  1. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
  2. 2016Structured Inference Networks for Nonlinear State Space Modelscited by 182
  3. 2015Deep Kalman Filterscited by 163
  4. 2020Dissecting Neural ODEscited by 150
  5. 2019An Attentive Survey of Attention Modelscited by 123
  6. 2019Attention in Natural Language Processingcited by 100
  7. 2019On Robustness of Neural Ordinary Differential Equationscited by 83
  8. 2020Learning Differential Equations that are Easy to Solvecited by 81
  9. 2020Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODEcited by 76
  10. 2017DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understandingcited by 71
  11. 2018Attention Models in Graphs: A Surveycited by 41
  12. 2015Describing Multimedia Content using Attention-based Encoder--Decoder Networkscited by 32
  13. 2018Dynamic Meta-Embeddings for Improved Sentence Representationscited by 15
  14. 2019Neural Ordinary Differential Equations for Semantic Segmentation of Individual Colon Glandscited by 15
  15. 2020Explainable Tensorized Neural Ordinary Differential Equations forArbitrary-step Time Series Predictioncited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 1; 2023: 3; 2024: 3; 2025: 4; 2026: 1.

15 papers.

Sort:

  1. 2024Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Datacited by 35
  2. 2021Attentive Neural Controlled Differential Equations for Time-series Classification and Forecastingcited by 28
  3. 2021LT-OCF: Learnable-Time ODE-based Collaborative Filteringcited by 21
  4. 2025Comprehensive Review of Neural Differential Equations for Time Series Analysiscited by 20
  5. 2024Unleashing The Power of Pre-Trained Language Models for Irregularly Sampled Time Seriescited by 10
  6. 2021LightMove: A Lightweight Next-POI Recommendation for Taxicab Rooftop Advertisingcited by 9
  7. 2023Learnable Path in Neural Controlled Differential Equationscited by 8
  8. 2024DualDynamics: Synergizing Implicit and Explicit Methods for Robust Irregular Time Series Analysiscited by 4
  9. 2025Counterfactual Probabilistic Diffusion with Expert Modelscited by 4
  10. 2023Long-term Time Series Forecasting based on Decomposition and Neural Ordinary Differential Equationscited by 1
  11. 2025Robust Filter Attention: Self-Attention as Precision-Weighted State Estimationcited by 1
  12. 2022Incorporating Interactive Facts for Stock Selection via Neural Recursive ODEscited by 0
3 more
  1. 2023Operator-learning-inspired Modeling of Neural Ordinary Differential Equationscited by 0
  2. 2025TANDEM: Temporal Attention-guided Neural Differential Equations for Missingness in Time Series Classificationcited by 0
  3. 2026Continuous-Time Machine Learning: A Unified Mathematical Perspectivecited by 0