# Temporal Transformer Networks: Joint Learning of Invariant and Discriminative Time Warping

arXiv: https://arxiv.org/abs/1906.05947 · HTML: https://mlsanity.com/paper/1906.05947/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2019.

## Built on by (most cited first, 21 of 21)

- [Warpformer: A Multi-scale Modeling Approach for Irregular Clinical Time Series](https://mlsanity.com/paper/2306.09368/) (2023)
- [Learning Discriminative Prototypes with Dynamic Time Warping](https://mlsanity.com/paper/2103.09458/) (2021)
- [Uncertainty-DTW for Time Series and Sequences](https://mlsanity.com/paper/2211.00005/) (2022)
- [Closed-Form Diffeomorphic Transformations for Time Series Alignment](https://mlsanity.com/paper/2206.08107/) (2022)
- [Deep Declarative Dynamic Time Warping for End-to-End Learning of Alignment Paths](https://mlsanity.com/paper/2303.10778/) (2023)
- [1D-Convolutional transformer for Parkinson disease diagnosis from gait](https://mlsanity.com/paper/2311.03177/) (2023)
- [Shifting the Paradigm: A Diffeomorphism Between Time Series Data Manifolds for Achieving Shift-Invariancy in Deep Learning](https://mlsanity.com/paper/2502.19921/) (2025)
- [Supervised deep learning of elastic SRV distances on the shape space of curves](https://mlsanity.com/paper/2101.04929/) (2021)
- [Residual Networks as Flows of Velocity Fields for Diffeomorphic Time Series Alignment](https://mlsanity.com/paper/2106.11911/) (2021)
- [Riemannian Self-Attention Mechanism for SPD Networks](https://mlsanity.com/paper/2311.16738/) (2023)
- [Towards Tokenized Human Dynamics Representation](https://mlsanity.com/paper/2111.11433/) (2021)
- [DeepFRC: An End-to-End Deep Learning Model for Functional Registration and Classification](https://mlsanity.com/paper/2501.18116/) (2025)
- [Diffeomorphic Temporal Alignment Nets for Time-series Joint Alignment and Averaging](https://mlsanity.com/paper/2502.06591/) (2025)
- [Spatial Transformer K-Means](https://mlsanity.com/paper/2202.07829/) (2022)
- [SrvfNet: A Generative Network for Unsupervised Multiple Diffeomorphic Shape Alignment](https://mlsanity.com/paper/2104.13449/) (2021)
- [DreamNet: A Deep Riemannian Network based on SPD Manifold Learning for Visual Classification](https://mlsanity.com/paper/2206.07967/) (2022)
- [Joint Alignment of Multivariate Quasi-Periodic Functional Data Using Deep Learning](https://mlsanity.com/paper/2312.09422/) (2023)
- [Deep Time Warping for Multiple Time Series Alignment](https://mlsanity.com/paper/2502.16324/) (2025)
- [Ground Reaction Force Estimation via Time-aware Knowledge Distillation](https://mlsanity.com/paper/2506.10265/) (2025)
- [InceptoFormer: A Multi-Signal Neural Framework for Parkinson's Disease Severity Evaluation from Gait](https://mlsanity.com/paper/2508.04540/) (2025)
- [Selective Correlation Based Knowledge Distillation for Ground Reaction Force Estimation](https://mlsanity.com/paper/2605.00888/) (2026)

## Abstract (arXiv)

Many time-series classification problems involve developing metrics that are invariant to temporal misalignment. In human activity analysis, temporal misalignment arises due to various reasons including differing initial phase, sensor sampling rates, and elastic time-warps due to subject-specific biomechanics. Past work in this area has only looked at reducing intra-class variability by elastic temporal alignment. In this paper, we propose a hybrid model-based and data-driven approach to learn warping functions that not just reduce intra-class variability, but also increase inter-class separation. We call this a temporal transformer network (TTN). TTN is an interpretable differentiable module, which can be easily integrated at the front end of a classification network. The module is capable of reducing intra-class variance by generating input-dependent warping functions which lead to rate-robust representations. At the same time, it increases inter-class variance by learning warping functions that are more discriminative. We show improvements over strong baselines in 3D action recognition on challenging datasets using the proposed framework. The improvements are especially pronounced when training sets are smaller.