# Learning to Exploit Invariances in Clinical Time-Series Data using Sequence Transformer Networks

arXiv: https://arxiv.org/abs/1808.06725 · HTML: https://mlsanity.com/paper/1808.06725/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Aug 2018.

## Built on by (most cited first, 10 of 10)

- [Gated Transformer Networks for Multivariate Time Series Classification](https://mlsanity.com/paper/2103.14438/) (2021)
- [Mixing Up Contrastive Learning: Self-Supervised Representation Learning for Time Series](https://mlsanity.com/paper/2203.09270/) (2022)
- [FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification](https://mlsanity.com/paper/2302.09818/) (2023)
- [Closed-Form Diffeomorphic Transformations for Time Series Alignment](https://mlsanity.com/paper/2206.08107/) (2022)
- [Shifting the Paradigm: A Diffeomorphism Between Time Series Data Manifolds for Achieving Shift-Invariancy in Deep Learning](https://mlsanity.com/paper/2502.19921/) (2025)
- [Enhancing Transformer Efficiency for Multivariate Time Series Classification](https://mlsanity.com/paper/2203.14472/) (2022)
- [Finding Short Signals in Long Irregular Time Series with Continuous-Time Attention Policy Networks](https://mlsanity.com/paper/2302.04052/) (2023)
- [SrvfNet: A Generative Network for Unsupervised Multiple Diffeomorphic Shape Alignment](https://mlsanity.com/paper/2104.13449/) (2021)
- [CaSS: A Channel-aware Self-supervised Representation Learning Framework for Multivariate Time Series Classification](https://mlsanity.com/paper/2203.04298/) (2022)
- [Joint Alignment of Multivariate Quasi-Periodic Functional Data Using Deep Learning](https://mlsanity.com/paper/2312.09422/) (2023)

## Abstract (arXiv)

Recently, researchers have started applying convolutional neural networks (CNNs) with one-dimensional convolutions to clinical tasks involving time-series data. This is due, in part, to their computational efficiency, relative to recurrent neural networks and their ability to efficiently exploit certain temporal invariances, (e.g., phase invariance). However, it is well-established that clinical data may exhibit many other types of invariances (e.g., scaling). While preprocessing techniques, (e.g., dynamic time warping) may successfully transform and align inputs, their use often requires one to identify the types of invariances in advance. In contrast, we propose the use of Sequence Transformer Networks, an end-to-end trainable architecture that learns to identify and account for invariances in clinical time-series data. Applied to the task of predicting in-hospital mortality, our proposed approach achieves an improvement in the area under the receiver operating characteristic curve (AUROC) relative to a baseline CNN (AUROC=0.851 vs. AUROC=0.838). Our results suggest that a variety of valuable invariances can be learned directly from the data.