# Shifting the Paradigm: A Diffeomorphism Between Time Series Data Manifolds for Achieving Shift-Invariancy in Deep Learning

arXiv: https://arxiv.org/abs/2502.19921 · HTML: https://mlsanity.com/paper/2502.19921/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 83 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [Why do deep convolutional networks generalize so poorly to small image transformations?](https://mlsanity.com/paper/1805.12177/) (2018)
- [CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients](https://mlsanity.com/paper/2005.13249/) (2020)
- [Equivariance with Learned Canonicalization Functions](https://mlsanity.com/paper/2211.06489/) (2022)
- [Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap](https://mlsanity.com/paper/2203.13457/) (2022)
- [Benign Overfitting in Two-layer Convolutional Neural Networks](https://mlsanity.com/paper/2202.06526/) (2022)
- [Adversarial vulnerability for any classifier](https://mlsanity.com/paper/1802.08686/) (2018)
- [Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?](https://mlsanity.com/paper/2106.02890/) (2021)
- [Truly shift-invariant convolutional neural networks](https://mlsanity.com/paper/2011.14214/) (2020)
- [Convolutional Kernel Networks](https://mlsanity.com/paper/1406.3332/) (2014)
- [HOLMES: Health OnLine Model Ensemble Serving for Deep Learning Models in Intensive Care Units](https://mlsanity.com/paper/2008.04063/) (2020)
- [Equivariant Adaptation of Large Pretrained Models](https://mlsanity.com/paper/2310.01647/) (2023)
- [TROIKA: A General Framework for Heart Rate Monitoring Using Wrist-Type Photoplethysmographic Signals During Intensive Physical Exercise](https://mlsanity.com/paper/1409.5181/) (2014)
- [What Makes Good Contrastive Learning on Small-Scale Wearable-based Tasks?](https://mlsanity.com/paper/2202.05998/) (2022)
- [Spatial-frequency channels, shape bias, and adversarial robustness](https://mlsanity.com/paper/2309.13190/) (2023)
- [Towards Open Respiratory Acoustic Foundation Models: Pretraining and Benchmarking](https://mlsanity.com/paper/2406.16148/) (2024)
- [Temporal Transformer Networks: Joint Learning of Invariant and Discriminative Time Warping](https://mlsanity.com/paper/1906.05947/) (2019)
- [Learnable Polyphase Sampling for Shift Invariant and Equivariant Convolutional Networks](https://mlsanity.com/paper/2210.08001/) (2022)
- [Finding Order in Chaos: A Novel Data Augmentation Method for Time Series in Contrastive Learning](https://mlsanity.com/paper/2309.13439/) (2023)
- [Group Equivariant Subsampling](https://mlsanity.com/paper/2106.05886/) (2021)
- [Alias-Free Convnets: Fractional Shift Invariance via Polynomial Activations](https://mlsanity.com/paper/2303.08085/) (2023)
- [What the F-measure doesn't measure: Features, Flaws, Fallacies and Fixes](https://mlsanity.com/paper/1503.06410/) (2015)
- [Learning to Exploit Invariances in Clinical Time-Series Data using Sequence Transformer Networks](https://mlsanity.com/paper/1808.06725/) (2018)

## Built on by (most cited first, 5 of 5)

- [STA-GANN: A Valid and Generalizable Spatio-Temporal Kriging Approach](https://mlsanity.com/paper/2508.16161/) (2025)
- [Alias-Free ViT: Fractional Shift Invariance via Linear Attention](https://mlsanity.com/paper/2510.22673/) (2025)
- [CANet: ChronoAdaptive Network for Enhanced Long-Term Time Series Forecasting under Non-Stationarity](https://mlsanity.com/paper/2504.17913/) (2025)
- [Learning Without Augmenting: Unsupervised Time Series Representation Learning via Frame Projections](https://mlsanity.com/paper/2510.22655/) (2025)
- [Unconditional flow-based time series generation with equivariance-regularised latent spaces](https://mlsanity.com/paper/2601.22848/) (2026)

## Abstract (arXiv)

Deep learning models lack shift invariance, making them sensitive to input shifts that cause changes in output. While recent techniques seek to address this for images, our findings show that these approaches fail to provide shift-invariance in time series, where the data generation mechanism is more challenging due to the interaction of low and high frequencies. Worse, they also decrease performance across several tasks. In this paper, we propose a novel differentiable bijective function that maps samples from their high-dimensional data manifold to another manifold of the same dimension, without any dimensional reduction. Our approach guarantees that samples -- when subjected to random shifts -- are mapped to a unique point in the manifold while preserving all task-relevant information without loss. We theoretically and empirically demonstrate that the proposed transformation guarantees shift-invariance in deep learning models without imposing any limits to the shift. Our experiments on six time series tasks with state-of-the-art methods show that our approach consistently improves the performance while enabling models to achieve complete shift-invariance without modifying or imposing restrictions on the model's topology. The source code is available on hrefhttps://github.com/eth-siplab/Shifting-the-ParadigmGitHub.