# Nystr"om Attention Matches Full Attention for Cross-Sectional Stock Prediction

arXiv: https://arxiv.org/abs/2609.08106 · HTML: https://mlsanity.com/paper/2609.08106/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 15 references that are in the map.
- First posted on arXiv 8 Sep 2026.

## Builds on (most cited first, 9 of 9)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attention](https://mlsanity.com/paper/2102.03902/) (2021)
- [Synthesizer: Rethinking Self-Attention in Transformer Models](https://mlsanity.com/paper/2005.00743/) (2020)
- [MASTER: Market-Guided Stock Transformer for Stock Price Forecasting](https://mlsanity.com/paper/2312.15235/) (2023)
- [Temporal and Heterogeneous Graph Neural Network for Financial Time Series Prediction](https://mlsanity.com/paper/2305.08740/) (2023)
- [LSR-IGRU: Stock Trend Prediction Based on Long Short-Term Relationships and Improved GRU](https://mlsanity.com/paper/2409.08282/) (2024)
- [FinMamba: Market-Aware Graph Enhanced Multi-Level Mamba for Stock Movement Prediction](https://mlsanity.com/paper/2502.06707/) (2025)
- [S3G: Stock State Space Graph for Enhanced Stock Trend Prediction](https://mlsanity.com/paper/2603.24236/) (2026)

## Abstract (arXiv)

MASTER's inter-stock multi-head attention -- the module responsible for modeling cross-sectional stock relationships -- accounts for 42.5% of model parameters and 25% of predictive value. We systematically decompose this module and uncover a surprising structure: the learned attention is near-uniform (perplexity 278/300), yet forcing exact uniformity eliminates all cross-sectional discrimination. Spectral analysis resolves this paradox: the deviation from uniformity is low-rank (effective rank ~65, top-10 modes capture 96.5% of energy), explaining why sparse approximations consistently fail while Nystrom low-rank attention (m=32 landmarks) matches full O(N^2) attention at O(mN) cost -- certified equivalent via TOST at both N=300 (5 seeds, Rank IC p=0.003) and N=800 (10 seeds, Rank IC p=0.034). Additional findings include: (i) attention anti-correlates with return similarity (Spearman rho = -0.614; on the industry-labeled subset, -0.645 unconditionally and -0.627 after controlling for industry, beta, and volatility), suggesting complementarity-seeking rather than correlation mining; (ii) all graph-based alternatives degrade performance, with hard masking worse than complete module removal; and (iii) at N ~ 3,500 with adapted architectures, no cross-stock module (GCN, Nystrom, or MASTER-style pipeline) significantly outperforms a per-stock LSTM baseline (n=4 seeds), indicating that the benefits observed at smaller scales do not trivially transfer. These results establish that the inter-stock attention's value resides in a compressible, dynamic, near-global redistribution that rewards low-rank approximation but resists sparsification.