# Is Sliding Window All You Need? An Open Framework for Long-Sequence Recommendation

arXiv: https://arxiv.org/abs/2604.12372 · HTML: https://mlsanity.com/paper/2604.12372/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 21 references that are in the map.
- First posted on arXiv 14 Apr 2026.

## Builds on (most cited first, 15 of 15)

- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens](https://mlsanity.com/paper/2402.13753/) (2024)
- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training](https://mlsanity.com/paper/2309.10400/) (2023)
- [GPT4Rec: A Generative Framework for Personalized Recommendation and User Interests Interpretation](https://mlsanity.com/paper/2304.03879/) (2023)
- [Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems](https://mlsanity.com/paper/1909.02107/) (2019)
- [Free and Customizable Code Documentation with LLMs: A Fine-Tuning Approach](https://mlsanity.com/paper/2412.00726/) (2024)
- [Single-Pass Pivot Algorithm for Correlation Clustering. Keep it simple!](https://mlsanity.com/paper/2305.13560/) (2023)
- [On the Consistency of Maximum Likelihood Estimation of Probabilistic Principal Component Analysis](https://mlsanity.com/paper/2311.05046/) (2023)
- [MM-PoE: Multiple Choice Reasoning via. Process of Elimination using Multi-Modal Models](https://mlsanity.com/paper/2412.07148/) (2024)
- [Sliding Window Training -- Utilizing Historical Recommender Systems Data for Foundation Models](https://mlsanity.com/paper/2409.14517/) (2024)
- [Time-Constrained Recommendations: Reinforcement Learning Strategies for E-Commerce](https://mlsanity.com/paper/2512.13726/) (2025)
- [The trade-offs of model size in large recommendation models : A 10000 × compressed criteo-tb DLRM model (100 GB parameters to mere 10MB)](https://mlsanity.com/paper/2207.10731/) (2022)
- [PixRec: Leveraging Visual Context for Next-Item Prediction in Sequential Recommendation](https://mlsanity.com/paper/2601.06458/) (2026)

## Abstract (arXiv)

Long interaction histories are central to modern recommender systems, yet training with long sequences is often dismissed as impractical under realistic memory and latency budgets. This work demonstrates that it is not only practical but also effective-at academic scale. We release a complete, end-to-end framework that implements industrial-style long-sequence training with sliding windows, including all data processing, training, and evaluation scripts. Beyond reproducing prior gains, we contribute two capabilities missing from earlier reports: (i) a runtime-aware ablation study that quantifies the accuracy-compute frontier across windowing regimes and strides, and (ii) a novel k-shift embedding layer that enables million-scale vocabularies on commodity GPUs with negligible accuracy loss. Our implementation trains reliably on modest university clusters while delivering competitive retrieval quality (e.g., up to +6.04% MRR and +6.34% Recall@10 on Retailrocket) with ~ 4 ×  training-time overheads. By packaging a robust pipeline, reporting training time costs, and introducing an embedding mechanism tailored for low-resource settings, we transform long-sequence training from a closed, industrial technique into a practical, open, and extensible methodology for the community.