# Segment, Shuffle, and Stitch: A Simple Layer for Improving Time-Series Representations

arXiv: https://arxiv.org/abs/2405.20082 · HTML: https://mlsanity.com/paper/2405.20082/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 53 references that are in the map.
- First posted on arXiv 30 May 2024.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting](https://mlsanity.com/paper/2007.02842/) (2020)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [The UEA multivariate time series classification archive, 2018](https://mlsanity.com/paper/1811.00075/) (2018)
- [SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction](https://mlsanity.com/paper/2106.09305/) (2021)
- [Fast Differentiable Sorting and Ranking](https://mlsanity.com/paper/2002.08871/) (2020)
- [CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients](https://mlsanity.com/paper/2005.13249/) (2020)
- [Time-Series Anomaly Detection Service at Microsoft](https://mlsanity.com/paper/1906.03821/) (2019)
- [An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networks](https://mlsanity.com/paper/2007.15951/) (2020)
- [Differentiable Ranks and Sorting using Optimal Transport](https://mlsanity.com/paper/1905.11885/) (2019)
- [Self-supervised ECG Representation Learning for Emotion Recognition](https://mlsanity.com/paper/2002.03898/) (2020)
- [ContiFormer: Continuous-Time Transformer for Irregular Time Series Modeling](https://mlsanity.com/paper/2402.10635/) (2024)
- [Towards Spatio-Temporal Aware Traffic Time Series Forecasting--Full Version](https://mlsanity.com/paper/2203.15737/) (2022)
- [Time Series Contrastive Learning with Information-Aware Augmentations](https://mlsanity.com/paper/2303.11911/) (2023)
- [Soft Contrastive Learning for Time Series](https://mlsanity.com/paper/2312.16424/) (2023)
- [Conditional Local Convolution for Spatio-temporal Meteorological Forecasting](https://mlsanity.com/paper/2101.01000/) (2021)
- [Dynamic Sparse Network for Time Series Classification: Learning What to "see''](https://mlsanity.com/paper/2212.09840/) (2022)
- [Region-Disentangled Diffusion Model for High-Fidelity PPG-to-ECG Translation](https://mlsanity.com/paper/2308.13568/) (2023)
- [Adaptive Graph Spatial-Temporal Transformer Network for Traffic Flow Forecasting](https://mlsanity.com/paper/2207.05064/) (2022)
- [Sub-Seasonal Climate Forecasting via Machine Learning: Challenges, Analysis, and Advances](https://mlsanity.com/paper/2006.07972/) (2020)
- [Self-Supervised Contrastive Learning for Long-term Forecasting](https://mlsanity.com/paper/2402.02023/) (2024)
- [Towards Diverse and Coherent Augmentation for Time-Series Forecasting](https://mlsanity.com/paper/2303.14254/) (2023)

## Built on by (most cited first, 6 of 6)

- [Thicker and Quicker: A Jumbo Token for Fast Plain Vision Transformers](https://mlsanity.com/paper/2502.15021/) (2025)
- [Universal Domain Adaptation Benchmark for Time Series Data Representation](https://mlsanity.com/paper/2505.17899/) (2025)
- [Masked Training for Robust Arrhythmia Detection from Digitalized Multiple Layout ECG Images](https://mlsanity.com/paper/2508.09165/) (2025)
- [Diversity Augmentation of Dynamic User Preference Data for Boosting Personalized Text Summarizers](https://mlsanity.com/paper/2510.10082/) (2025)
- [Graph-Based Learning of Spectro-Topographical EEG Representations with Gradient Alignment for Brain-Computer Interfaces](https://mlsanity.com/paper/2512.07820/) (2025)
- [A Comprehensive Benchmark of Source-Free Universal Domain Adaptation on Time Series Representations](https://mlsanity.com/paper/2609.39810/) (2026)

## Abstract (arXiv)

Existing approaches for learning representations of time-series keep the temporal arrangement of the time-steps intact with the presumption that the original order is the most optimal for learning. However, non-adjacent sections of real-world time-series may have strong dependencies. Accordingly, we raise the question: Is there an alternative arrangement for time-series which could enable more effective representation learning? To address this, we propose a simple plug-and-play neural network layer called Segment, Shuffle, and Stitch (S3) designed to improve representation learning in time-series models. S3 works by creating non-overlapping segments from the original sequence and shuffling them in a learned manner that is optimal for the task at hand. It then re-attaches the shuffled segments back together and performs a learned weighted sum with the original input to capture both the newly shuffled sequence along with the original sequence. S3 is modular and can be stacked to achieve different levels of granularity, and can be added to many forms of neural architectures including CNNs or Transformers with negligible computation overhead. Through extensive experiments on several datasets and state-of-the-art baselines, we show that incorporating S3 results in significant improvements for the tasks of time-series classification, forecasting, and anomaly detection, improving performance on certain datasets by up to 68%. We also show that S3 makes the learning more stable with a smoother training loss curve and loss landscape compared to the original baseline. The code is available at https://github.com/shivam-grover/S3-TimeSeries.