mlsanityData to 7 Oct 2026

Multi-Time Attention Networks for Irregularly Sampled Time Series

arXiv 2101.10318 ↗First posted 25 Jan 2021cs.LG cs.AIcited by 234

Not on the map. Why?

Abstract (arXiv)

Irregular sampling occurs in many time series modeling applications where it presents a significant challenge to standard deep learning models. This work is motivated by the analysis of physiological time series data in electronic health records, which are sparse, irregularly sampled, and multivariate. In this paper, we propose a new deep learning framework for this setting that we call Multi-Time Attention Networks. Multi-Time Attention Networks learn an embedding of continuous-time values and use an attention mechanism to produce a fixed-length representation of a time series containing a variable number of observations. We investigate the performance of this framework on interpolation and classification tasks using multiple datasets. Our results show that the proposed approach performs as well or better than a range of baseline and recently proposed models while offering significantly faster training times than current state-of-the-art methods.

Key facts. Cited by 234 papers in the map (data through 7 Oct 2026). Builds on 20 of its 38 references that are in the map. First posted on arXiv 25 Jan 2021.

Builds on

20 of 38 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  5. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 548
  6. 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 230
  7. 2019Time2Vec: Learning a Vector Representation of Timecited by 208
  8. 2019Latent ODEs for Irregularly-Sampled Time Seriescited by 182
  9. 2019Set Functions for Time Seriescited by 137
  10. 2017Attend and Diagnose: Clinical Time Series Analysis using Attention Modelscited by 105
  11. 2016Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequencescited by 98
  12. 2019Interpolation-Prediction Networks for Irregularly Sampled Time Seriescited by 89
8 more
  1. 2017Estimating Missing Data in Temporal Data Streams Using Multi-directional Recurrent Neural Networkscited by 86
  2. 2016Modeling Missing Data in Clinical Time Series with RNNscited by 80
  3. 2019Self-attention with Functional Time Representation Learningcited by 52
  4. 2017Learning to Detect Sepsis with a Multitask Gaussian Process RNN Classifiercited by 47
  5. 2017Discrete Event, Continuous Time RNNscited by 31
  6. 2016A scalable end-to-end Gaussian process adapter for irregularly sampled time series classificationcited by 27
  7. 2018Medical Concept Embedding with Time-Aware Attentioncited by 11
  8. 2019Temporal Self-Attention Network for Medical Concept Embeddingcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 15; 2023: 27; 2024: 54; 2025: 45; 2026: 83.

234 papers.

Sort:

By area:

  1. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  2. 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputationcited by 577
  3. 2021Graph-Guided Network for Irregularly Sampled Multivariate Time Seriescited by 110
  4. 2021Modeling Irregular Time Series with Continuous Recurrent Unitscited by 101
  5. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  6. 2023Time Series as Images: Vision Transformer for Irregularly Sampled Time Seriescited by 79
  7. 2024FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusioncited by 67
  8. 2021Closed-form Continuous-time Neural Modelscited by 65
  9. 2022Learning to Reconstruct Missing Data from Spatiotemporal Graphs with Sparse Observationscited by 64
  10. 2023PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputationcited by 62
  11. 2024ContiFormer: Continuous-Time Transformer for Irregular Time Series Modelingcited by 61
  12. 2023Asynchrony-Robust Collaborative Perception via Bird's Eye View Flowcited by 58
38 more
  1. 2023Warpformer: A Multi-scale Modeling Approach for Irregular Clinical Time Seriescited by 54
  2. 2025Rethinking Irregular Time Series Forecasting: A Simple yet Effective Baselinecited by 54
  3. 2023On the Constrained Time-Series Generation Problemcited by 52
  4. 2023Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Predictioncited by 47
  5. 2022TACTiS: Transformer-Attentional Copulas for Time Seriescited by 38
  6. 2024Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Datacited by 35
  7. 2022Improving Medical Predictions by Irregular Multimodal Electronic Health Records Modelingcited by 33
  8. 2024Universal Time-Series Representation Learning: A Surveycited by 28
  9. 2021Towards Generating Real-World Time Series Datacited by 26
  10. 2023Provably Convergent Schr"odinger Bridge with Applications to Probabilistic Time Series Imputationcited by 26
  11. 2021NRTSI: Non-Recurrent Time Series Imputationcited by 25
  12. 2024Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Modelscited by 23
  13. 2024ColaCare: Enhancing Electronic Health Record Modeling through Large Language Model-Driven Multi-Agent Collaborationcited by 23
  14. 2023Probabilistic Imputation for Time-series Classification with Missing Datacited by 21
  15. 2024Trajectory Flow Matching with Applications to Clinical Time Series Modelingcited by 21
  16. 2025Comprehensive Review of Neural Differential Equations for Time Series Analysiscited by 20
  17. 2023Irregular Traffic Time Series Forecasting Based on Asynchronous Spatio-Temporal Graph Convolutional Networkcited by 19
  18. 2025Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictionscited by 18
  19. 2022Latent Neural ODEs with Sparse Bayesian Multiple Shootingcited by 17
  20. 2023DuETT: Dual Event Time Transformer for Electronic Health Recordscited by 17
  21. 2023Time Series Continuous Modeling for Imputation and Forecasting with Implicit Neural Representationscited by 16
  22. 2023Latent SDEs on Homogeneous Spacescited by 16
  23. 2024On Expert Estimation in Hierarchical Mixture of Experts: Beyond Softmax Gating Functionscited by 16
  24. 2025Time-IMM: A Dataset and Benchmark for Irregular Multimodal Multivariate Time Seriescited by 16
  25. 2024Graph-based Forecasting with Missing Data through Spatiotemporal Downsamplingcited by 15
  26. 2025HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecastingcited by 14
  27. 2024Zero-shot Imputation with Foundation Inference Models for Dynamical Systemscited by 13
  28. 2022Temporal Label Smoothing for Early Event Predictioncited by 12
  29. 2024Causality-Aware Spatiotemporal Graph Neural Networks for Spatiotemporal Time Series Imputationcited by 11
  30. 2022Stop&Hop: Early Classification of Irregular Time Seriescited by 10
  31. 2024Self-Supervised Learning of Time Series Representation via Diffusion Process and Imputation-Interpolation-Forecasting Maskcited by 10
  32. 2024Unleashing The Power of Pre-Trained Language Models for Irregularly Sampled Time Seriescited by 10
  33. 2024TimeCHEAT: A Channel Harmony Strategy for Irregularly Sampled Multivariate Time Series Analysiscited by 10
  34. 2023FineEHR: Refine Clinical Note Representations to Improve Mortality Predictioncited by 9
  35. 2024STD-PLM: Understanding Both Spatial and Temporal Properties of Spatial-Temporal Data with PLMcited by 9
  36. 2024Amortized Control of Continuous State Space Feynman-Kac Model for Irregular Time Seriescited by 9
  37. 2024A Systematic Literature Review of Spatio-Temporal Graph Neural Network Models for Time Series Forecasting and Classificationcited by 9
  38. 2025MAESTRO : Adaptive Sparse Attention and Robust Learning for Multimodal Dynamic Time Seriescited by 9

50 most cited of 234 All 234 (JSON)