# Warpformer: A Multi-scale Modeling Approach for Irregular Clinical Time Series

arXiv: https://arxiv.org/abs/2306.09368 · HTML: https://mlsanity.com/paper/2306.09368/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 62 references that are in the map.
- First posted on arXiv 14 Jun 2023.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Learning Deep Transformer Models for Machine Translation](https://mlsanity.com/paper/1906.01787/) (2019)
- [Multitask learning and benchmarking with clinical time series data](https://mlsanity.com/paper/1703.07771/) (2017)
- [Soft-DTW: a Differentiable Loss Function for Time-Series](https://mlsanity.com/paper/1703.01541/) (2017)
- [Multi-Time Attention Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/2101.10318/) (2021)
- [The Evolved Transformer](https://mlsanity.com/paper/1901.11117/) (2019)
- [Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures](https://mlsanity.com/paper/2207.01186/) (2022)
- [Set Functions for Time Series](https://mlsanity.com/paper/1909.12064/) (2019)
- [MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-III](https://mlsanity.com/paper/1907.08322/) (2019)
- [Graph-Guided Network for Irregularly Sampled Multivariate Time Series](https://mlsanity.com/paper/2110.05357/) (2021)
- [Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences](https://mlsanity.com/paper/1610.09513/) (2016)
- [Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series](https://mlsanity.com/paper/2107.14293/) (2021)
- [Interpolation-Prediction Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/1909.07782/) (2019)
- [Predicting Path Failure In Time-Evolving Graphs](https://mlsanity.com/paper/1905.03994/) (2019)
- [Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecasting](https://mlsanity.com/paper/2206.04038/) (2022)
- [AdaCare: Explainable Clinical Health Status Representation Learning via Scale-Adaptive Feature Extraction and Recalibration](https://mlsanity.com/paper/1911.12205/) (2019)
- [RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Data](https://mlsanity.com/paper/1807.08820/) (2018)
- [Discrete Event, Continuous Time RNNs](https://mlsanity.com/paper/1710.04110/) (2017)
- [MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records](https://mlsanity.com/paper/2102.02340/) (2021)
- [Temporal Transformer Networks: Joint Learning of Invariant and Discriminative Time Warping](https://mlsanity.com/paper/1906.05947/) (2019)

## Built on by (most cited first, 30 of 55)

- [Rethinking Irregular Time Series Forecasting: A Simple yet Effective Baseline](https://mlsanity.com/paper/2505.11250/) (2025)
- [Irregular Traffic Time Series Forecasting Based on Asynchronous Spatio-Temporal Graph Convolutional Network](https://mlsanity.com/paper/2308.16818/) (2023)
- [Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions](https://mlsanity.com/paper/2507.00191/) (2025)
- [HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting](https://mlsanity.com/paper/2505.17431/) (2025)
- [Unleashing The Power of Pre-Trained Language Models for Irregularly Sampled Time Series](https://mlsanity.com/paper/2408.08328/) (2024)
- [ElasTST: Towards Robust Varied-Horizon Forecasting with Elastic Time-Series Transformer](https://mlsanity.com/paper/2411.01842/) (2024)
- [Bridging Time and Frequency: A Joint Modeling Framework for Irregular Multivariate Time Series Forecasting](https://mlsanity.com/paper/2602.00582/) (2026)
- [A Co-training Approach for Noisy Time Series Learning](https://mlsanity.com/paper/2308.12551/) (2023)
- [SMART: Towards Pre-trained Missing-Aware Model for Patient Health Status Prediction](https://mlsanity.com/paper/2405.09039/) (2024)
- [ASTGI: Adaptive Spatio-Temporal Graph Interactions for Irregular Multivariate Time Series Forecasting](https://mlsanity.com/paper/2509.23313/) (2025)
- [CTPD: Cross-Modal Temporal Pattern Discovery for Enhanced Multimodal Electronic Health Records Analysis](https://mlsanity.com/paper/2411.00696/) (2024)
- [Revitalizing Canonical Pre-Alignment for Irregular Multivariate Time Series Forecasting](https://mlsanity.com/paper/2508.01971/) (2025)
- [Irregularity-Informed Time Series Analysis: Adaptive Modelling of Spatial and Temporal Dynamics](https://mlsanity.com/paper/2410.12257/) (2024)
- [Scalable Transformer for High Dimensional Multivariate Time Series Forecasting](https://mlsanity.com/paper/2408.04245/) (2024)
- [Large Language Model as a Universal Clinical Multi-task Decoder](https://mlsanity.com/paper/2406.12738/) (2024)
- [IMTS is Worth Time × Channel Patches: Visual Masked Autoencoders for Irregular Multivariate Time Series Prediction](https://mlsanity.com/paper/2505.22815/) (2025)
- [TimePoint: Accelerated Time Series Alignment via Self-Supervised Keypoint and Descriptor Learning](https://mlsanity.com/paper/2505.23475/) (2025)
- [MTM: A Multi-Scale Token Mixing Transformer for Irregular Multivariate Time Series Classification](https://mlsanity.com/paper/2509.17809/) (2025)
- [Machine Learning Approaches to Clinical Risk Prediction: Multi-Scale Temporal Alignment in Electronic Health Records](https://mlsanity.com/paper/2511.21561/) (2025)
- [MILM: Large Language Models for Multimodal Irregular Time Series with Informative Sampling](https://mlsanity.com/paper/2605.13711/) (2026)
- [Two-Stage Aggregation with Dynamic Local Attention for Irregular Time Series](https://mlsanity.com/paper/2311.07744/) (2023)
- [Evaluating Time-Series Training Dataset through Lens of Spectrum in Deep State Space Models](https://mlsanity.com/paper/2408.16261/) (2024)
- [WaveGNN: Integrating Graph Neural Networks and Transformers for Decay-Aware Classification of Irregular Clinical Time-Series](https://mlsanity.com/paper/2412.10621/) (2024)
- [Integrating Sequence and Image Modeling in Irregular Medical Time Series Through Self-Supervised Learning](https://mlsanity.com/paper/2502.06134/) (2025)
- [Mixing It Up: Exploring Mixer Networks for Irregular Multivariate Time Series Forecasting](https://mlsanity.com/paper/2502.11816/) (2025)
- [Towards Robust Real-World Multivariate Time Series Forecasting: A Unified Framework for Dependency, Asynchrony, and Missingness](https://mlsanity.com/paper/2506.08660/) (2025)
- [Beyond Observations: Reconstruction Error-Guided Irregularly Sampled Time Series Representation Learning](https://mlsanity.com/paper/2511.06854/) (2025)
- [Rethinking Large Language Models For Irregular Time Series Classification In Critical Care](https://mlsanity.com/paper/2601.16516/) (2026)
- [MM-ISTS: Cooperating Irregularly Sampled Time Series Forecasting with Multimodal Vision-Text LLMs](https://mlsanity.com/paper/2603.05997/) (2026)
- [One-Step Graph-Structured Neural Flows for Irregular Multivariate Time Series Classification](https://mlsanity.com/paper/2605.10179/) (2026)

## Abstract (arXiv)

Irregularly sampled multivariate time series are ubiquitous in various fields, particularly in healthcare, and exhibit two key characteristics: intra-series irregularity and inter-series discrepancy. Intra-series irregularity refers to the fact that time-series signals are often recorded at irregular intervals, while inter-series discrepancy refers to the significant variability in sampling rates among diverse series. However, recent advances in irregular time series have primarily focused on addressing intra-series irregularity, overlooking the issue of inter-series discrepancy. To bridge this gap, we present Warpformer, a novel approach that fully considers these two characteristics. In a nutshell, Warpformer has several crucial designs, including a specific input representation that explicitly characterizes both intra-series irregularity and inter-series discrepancy, a warping module that adaptively unifies irregular time series in a given scale, and a customized attention module for representation learning. Additionally, we stack multiple warping and attention modules to learn at different scales, producing multi-scale representations that balance coarse-grained and fine-grained signals for downstream tasks. We conduct extensive experiments on widely used datasets and a new large-scale benchmark built from clinical databases. The results demonstrate the superiority of Warpformer over existing state-of-the-art approaches.