# A Set-Sequence Model for Time Series

arXiv: https://arxiv.org/abs/2505.11243 · HTML: https://mlsanity.com/paper/2505.11243/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 16 May 2025.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks](https://mlsanity.com/paper/2005.11650/) (2020)
- [Hungry Hungry Hippos: Towards Language Modeling with State Space Models](https://mlsanity.com/paper/2212.14052/) (2022)
- [Hyena Hierarchy: Towards Larger Convolutional Language Models](https://mlsanity.com/paper/2302.10866/) (2023)
- [TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting](https://mlsanity.com/paper/2405.14616/) (2024)
- [TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables](https://mlsanity.com/paper/2402.19072/) (2024)
- [TSMixer: An All-MLP Architecture for Time Series Forecasting](https://mlsanity.com/paper/2303.06053/) (2023)
- [Frequency-domain MLPs are More Effective Learners in Time Series Forecasting](https://mlsanity.com/paper/2311.06184/) (2023)
- [Discrete Graph Structure Learning for Forecasting Multiple Time Series](https://mlsanity.com/paper/2101.06861/) (2021)
- [Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting](https://mlsanity.com/paper/1905.03806/) (2019)
- [High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes](https://mlsanity.com/paper/1910.03002/) (2019)
- [Set Functions for Time Series](https://mlsanity.com/paper/1909.12064/) (2019)
- [TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis](https://mlsanity.com/paper/2410.16032/) (2024)
- [Is Mamba Effective for Time Series Forecasting?](https://mlsanity.com/paper/2403.11144/) (2024)
- [Deep Factors for Forecasting](https://mlsanity.com/paper/1905.12417/) (2019)
- [Simple Hardware-Efficient Long Convolutions for Sequence Modeling](https://mlsanity.com/paper/2302.06646/) (2023)
- [Effectively Modeling Time Series with Simple Discrete State Spaces](https://mlsanity.com/paper/2303.09489/) (2023)
- [Fredformer: Frequency Debiased Transformer for Time Series Forecasting](https://mlsanity.com/paper/2406.09009/) (2024)
- [Deep Learning in Asset Pricing](https://mlsanity.com/paper/1904.00745/) (2019)
- [Deep Learning Statistical Arbitrage](https://mlsanity.com/paper/2106.04028/) (2021)

## Built on by (most cited first, 1 of 1)

- [AICO: Feature Significance Tests for Supervised Learning](https://mlsanity.com/paper/2506.23396/) (2025)

## Abstract (arXiv)

Many prediction problems across science and engineering, especially in finance and economics, involve large cross-sections of individual time series, where each unit (e.g., a loan, stock, or customer) is driven by unit-level features and latent cross-sectional dynamics. While sequence models have advanced per-unit temporal prediction, capturing cross-sectional effects often still relies on hand-crafted summary features. We propose Set-Sequence, a model that learns cross-sectional structure directly, enhancing expressivity and eliminating manual feature engineering. At each time step, a permutation-invariant Set module summarizes the unit set; a Sequence module then models each unit's dynamics conditioned on both its features and the learned summary. The architecture accommodates unaligned series, supports varying numbers of units at inference, integrates with standard sequence backbones (e.g., Transformers), and scales linearly in cross-sectional size. Across a synthetic contagion task and two large-scale real-world applications, equity portfolio optimization and loan risk prediction, Set-Sequence significantly outperforms strong baselines, delivering higher Sharpe ratios, improved AUCs, and interpretable cross-sectional summaries.