Is Sliding Window All You Need? An Open Framework for Long-Sequence Recommendation
Not on the map. Why?
Abstract (arXiv)
Long interaction histories are central to modern recommender systems, yet training with long sequences is often dismissed as impractical under realistic memory and latency budgets. This work demonstrates that it is not only practical but also effective-at academic scale. We release a complete, end-to-end framework that implements industrial-style long-sequence training with sliding windows, including all data processing, training, and evaluation scripts. Beyond reproducing prior gains, we contribute two capabilities missing from earlier reports: (i) a runtime-aware ablation study that quantifies the accuracy-compute frontier across windowing regimes and strides, and (ii) a novel k-shift embedding layer that enables million-scale vocabularies on commodity GPUs with negligible accuracy loss. Our implementation trains reliably on modest university clusters while delivering competitive retrieval quality (e.g., up to +6.04% MRR and +6.34% Recall@10 on Retailrocket) with ~ 4 × training-time overheads. By packaging a robust pipeline, reporting training time costs, and introducing an embedding mechanism tailored for low-resource settings, we transform long-sequence training from a closed, industrial technique into a practical, open, and extensible methodology for the community.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 14 Apr 2026.
Builds on
15 of 21 references.
References · 15
- 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 952
- 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 396
- 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 302
- 2024Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendationscited by 184
- 2023PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Trainingcited by 109
- 2023GPT4Rec: A Generative Framework for Personalized Recommendation and User Interests Interpretationcited by 101
- 2019Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systemscited by 68
- 2024Free and Customizable Code Documentation with LLMs: A Fine-Tuning Approachcited by 10
- 2023Single-Pass Pivot Algorithm for Correlation Clustering. Keep it simple!cited by 8
- 2023On the Consistency of Maximum Likelihood Estimation of Probabilistic Principal Component Analysiscited by 5
- 2024MM-PoE: Multiple Choice Reasoning via. Process of Elimination using Multi-Modal Modelscited by 5
- 2024Sliding Window Training -- Utilizing Historical Recommender Systems Data for Foundation Modelscited by 3
3 more
- 2025Time-Constrained Recommendations: Reinforcement Learning Strategies for E-Commercecited by 3
- 2022The trade-offs of model size in large recommendation models : A 10000 × compressed criteo-tb DLRM model (100 GB parameters to mere 10MB)cited by 2
- 2026PixRec: Leveraging Visual Context for Next-Item Prediction in Sequential Recommendationcited by 1
Built on by
Not cited yet.