# Time Elastic Neural Networks

arXiv: https://arxiv.org/abs/2405.17516 · HTML: https://mlsanity.com/paper/2405.17516/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 52 references that are in the map.
- First posted on arXiv 27 May 2024.

## Builds on (most cited first, 10 of 10)

- [Soft-DTW: a Differentiable Loss Function for Time-Series](https://mlsanity.com/paper/1703.01541/) (2017)
- [LSTM Fully Convolutional Networks for Time Series Classification](https://mlsanity.com/paper/1709.05206/) (2017)
- [Bake off redux: a review and experimental evaluation of recent time series classification algorithms](https://mlsanity.com/paper/2304.13029/) (2023)
- [Multivariate Time Series Classification with WEASEL+MUSE](https://mlsanity.com/paper/1711.11343/) (2017)
- [The Great Time Series Classification Bake Off: An Experimental Evaluation of Recently Proposed Algorithms. Extended Version](https://mlsanity.com/paper/1602.01711/) (2016)
- [NeuralWarp: Time-Series Similarity with Warping Networks](https://mlsanity.com/paper/1812.08306/) (2018)
- [Exact Mean Computation in Dynamic Time Warping Spaces](https://mlsanity.com/paper/1710.08937/) (2017)
- [Attention to Warp: Deep Metric Learning for Multivariate Time Series](https://mlsanity.com/paper/2103.15074/) (2021)
- [Times series averaging and denoising from a probabilistic perspective on time-elastic kernels](https://mlsanity.com/paper/1611.09194/) (2016)
- [Sparsification of the Alignment Path Search Space in Dynamic Time Warping](https://mlsanity.com/paper/1711.04453/) (2017)

## Abstract (arXiv)

We introduce and detail an atypical neural network architecture, called time elastic neural network (teNN), for multivariate time series classification. The novelty compared to classical neural network architecture is that it explicitly incorporates time warping ability, as well as a new way of considering attention. In addition, this architecture is capable of learning a dropout strategy, thus optimizing its own architecture.Behind the design of this architecture, our overall objective is threefold: firstly, we are aiming at improving the accuracy of instance based classification approaches that shows quite good performances as far as enough training data is available. Secondly we seek to reduce the computational complexity inherent to these methods to improve their scalability. Ideally, we seek to find an acceptable balance between these first two criteria. And finally, we seek to enhance the explainability of the decision provided by this kind of neural architecture.The experiment demonstrates that the stochastic gradient descent implemented to train a teNN is quite effective. To the extent that the selection of some critical meta-parameters is correct, convergence is generally smooth and fast.While maintaining good accuracy, we get a drastic gain in scalability by first reducing the required number of reference time series, i.e. the number of teNN cells required. Secondly, we demonstrate that, during the training process, the teNN succeeds in reducing the number of neurons required within each cell. Finally, we show that the analysis of the activation and attention matrices as well as the reference time series after training provides relevant information to interpret and explain the classification results.The comparative study that we have carried out and which concerns around thirty diverse and multivariate datasets shows that the teNN obtains results comparable to those of the state of the art, in particular similar to those of a network mixing LSTM and CNN architectures for example.