# Attend and Diagnose: Clinical Time Series Analysis using Attention Models

arXiv: https://arxiv.org/abs/1711.03905 · HTML: https://mlsanity.com/paper/1711.03905/

## Key facts

- Cited by 105 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2017.

## Built on by (most cited first, 30 of 105)

- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Multi-Time Attention Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/2101.10318/) (2021)
- [Deep Learning for Time Series Anomaly Detection: A Survey](https://mlsanity.com/paper/2211.05244/) (2022)
- [Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series](https://mlsanity.com/paper/2107.14293/) (2021)
- [Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement](https://mlsanity.com/paper/2301.03028/) (2023)
- [Explaining Time Series Predictions with Dynamic Masks](https://mlsanity.com/paper/2106.05303/) (2021)
- [Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey](https://mlsanity.com/paper/2302.02515/) (2023)
- [A General Survey on Attention Mechanisms in Deep Learning](https://mlsanity.com/paper/2203.14263/) (2022)
- [Ti-MAE: Self-Supervised Masked Time Series Autoencoders](https://mlsanity.com/paper/2301.08871/) (2023)
- [MTS-Mixers: Multivariate Time Series Forecasting via Factorized Temporal and Channel Mixing](https://mlsanity.com/paper/2302.04501/) (2023)
- [Transformers in Time-series Analysis: A Tutorial](https://mlsanity.com/paper/2205.01138/) (2022)
- [GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learning](https://mlsanity.com/paper/2207.09858/) (2022)
- [Temporal Dependencies in Feature Importance for Time Series Predictions](https://mlsanity.com/paper/2107.14317/) (2021)
- [Unifying Heterogeneous Electronic Health Records Systems via Text-Based Code Embedding](https://mlsanity.com/paper/2108.03625/) (2021)
- [WindowSHAP: An Efficient Framework for Explaining Time-series Classifiers based on Shapley Values](https://mlsanity.com/paper/2211.06507/) (2022)
- [Preformer: Predictive Transformer with Multi-Scale Segment-wise Correlations for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2202.11356/) (2022)
- [Multi-Label Clinical Time-Series Generation via Conditional GAN](https://mlsanity.com/paper/2204.04797/) (2022)
- [Explainable Deep Learning in Healthcare: A Methodological Survey from an Attribution View](https://mlsanity.com/paper/2112.02625/) (2021)
- [TiSAT: Time Series Anomaly Transformer](https://mlsanity.com/paper/2203.05167/) (2022)
- [Integrating Physiological Time Series and Clinical Notes with Transformer for Early Prediction of Sepsis](https://mlsanity.com/paper/2203.14469/) (2022)
- [Positional Encoding in Transformer-Based Time Series Models: A Survey](https://mlsanity.com/paper/2502.12370/) (2025)
- [Clairvoyance: A Pipeline Toolkit for Medical Time Series](https://mlsanity.com/paper/2310.18688/) (2023)
- [SCEHR: Supervised Contrastive Learning for Clinical Risk Prediction using Electronic Health Records](https://mlsanity.com/paper/2110.04943/) (2021)
- [Graph-Text Multi-Modal Pre-training for Medical Representation Learning](https://mlsanity.com/paper/2203.09994/) (2022)
- [Counterfactual Explanations for Time Series Forecasting](https://mlsanity.com/paper/2310.08137/) (2023)
- [A Systematic Review for Transformer-based Long-term Series Forecasting](https://mlsanity.com/paper/2310.20218/) (2023)
- [Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values](https://mlsanity.com/paper/2401.05800/) (2024)
- [Towards the Practical Utility of Federated Learning in the Medical Domain](https://mlsanity.com/paper/2207.03075/) (2022)
- [TCCT: Tightly-Coupled Convolutional Transformer on Time Series Forecasting](https://mlsanity.com/paper/2108.12784/) (2021)
- [SMART: Towards Pre-trained Missing-Aware Model for Patient Health Status Prediction](https://mlsanity.com/paper/2405.09039/) (2024)

## Abstract (arXiv)

With widespread adoption of electronic health records, there is an increased emphasis for predictive models that can effectively deal with clinical time-series data. Powered by Recurrent Neural Network (RNN) architectures with Long Short-Term Memory (LSTM) units, deep neural networks have achieved state-of-the-art results in several clinical prediction tasks. Despite the success of RNNs, its sequential nature prohibits parallelized computing, thus making it inefficient particularly when processing long sequences. Recently, architectures which are based solely on attention mechanisms have shown remarkable success in transduction tasks in NLP, while being computationally superior. In this paper, for the first time, we utilize attention models for clinical time-series modeling, thereby dispensing recurrence entirely. We develop the textitSAnD (Simply Attend and Diagnose) architecture, which employs a masked, self-attention mechanism, and uses positional encoding and dense interpolation strategies for incorporating temporal order. Furthermore, we develop a multi-task variant of textitSAnD to jointly infer models with multiple diagnosis tasks. Using the recent MIMIC-III benchmark datasets, we demonstrate that the proposed approach achieves state-of-the-art performance in all tasks, outperforming LSTM models and classical baselines with hand-engineered features.