# TimewarpVAE: Simultaneous Time-Warping and Representation Learning of Trajectories

arXiv: https://arxiv.org/abs/2310.16027 · HTML: https://mlsanity.com/paper/2310.16027/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 39 references that are in the map.
- First posted on arXiv 24 Oct 2023.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Soft-DTW: a Differentiable Loss Function for Time-Series](https://mlsanity.com/paper/1703.01541/) (2017)
- [On Implicit Regularization in β-VAEs](https://mlsanity.com/paper/2002.00041/) (2020)
- [Learning Discriminative Prototypes with Dynamic Time Warping](https://mlsanity.com/paper/2103.09458/) (2021)
- [Nonsmooth Analysis and Subgradient Methods for Averaging in Dynamic Time Warping Spaces](https://mlsanity.com/paper/1701.06393/) (2017)
- [Representing Spatial Trajectories as Distributions](https://mlsanity.com/paper/2210.01322/) (2022)
- [Neural Latent Aligner: Cross-trial Alignment for Learning Representations of Complex, Naturalistic Neural Data](https://mlsanity.com/paper/2308.06443/) (2023)

## Built on by (most cited first, 1 of 1)

- [Fast Gradient-Based Approximation of Continuous Dynamic Time Warping](https://mlsanity.com/paper/2610.13129/) (2026)

## Abstract (arXiv)

Human demonstrations of trajectories are an important source of training data for many machine learning problems. However, the difficulty of collecting human demonstration data for complex tasks makes learning efficient representations of those trajectories challenging. For many problems, such as for dexterous manipulation, the exact timings of the trajectories should be factored from their spatial path characteristics. In this work, we propose TimewarpVAE, a fully differentiable manifold-learning algorithm that incorporates Dynamic Time Warping (DTW) to simultaneously learn both timing variations and latent factors of spatial variation. We show how the TimewarpVAE algorithm learns appropriate time alignments and meaningful representations of spatial variations in handwriting and fork manipulation datasets. Our results have lower spatial reconstruction test error than baseline approaches and the learned low-dimensional representations can be used to efficiently generate semantically meaningful novel trajectories. We demonstrate the utility of our algorithm to generate novel high-speed trajectories for a robotic arm.