# SinkRec: Mitigating Semantic State Sink in Long Sequence Recommendation with Memory-Conditioned Gated Delta Networks

arXiv: https://arxiv.org/abs/2606.09888 · HTML: https://mlsanity.com/paper/2606.09888/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 36 references that are in the map.
- First posted on arXiv 3 Jun 2026.

## Builds on (most cited first, 30 of 34)

- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [Augmenting Language Models with Long-Term Memory](https://mlsanity.com/paper/2306.07174/) (2023)
- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction](https://mlsanity.com/paper/2006.05639/) (2020)
- [Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.09248/) (2019)
- [KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems](https://mlsanity.com/paper/2202.10842/) (2022)
- [Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models](https://mlsanity.com/paper/2601.07372/) (2026)
- [LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders](https://mlsanity.com/paper/2505.04421/) (2025)
- [RankMixer: Scaling Up Ranking Models in Industrial Recommenders](https://mlsanity.com/paper/2507.15551/) (2025)
- [TWIN: TWo-stage Interest Network for Lifelong User Behavior Modeling in CTR Prediction at Kuaishou](https://mlsanity.com/paper/2302.02352/) (2023)
- [Mamba4Rec: Towards Efficient Sequential Recommendation with Selective State Space Models](https://mlsanity.com/paper/2403.03900/) (2024)
- [Ultra-Sparse Memory Network](https://mlsanity.com/paper/2411.12364/) (2024)
- [Memory Caching: RNNs with Growing Memory](https://mlsanity.com/paper/2602.24281/) (2026)
- [Massive Memorization with Hundreds of Trillions of Parameters for Sequential Transducer Generative Recommenders](https://mlsanity.com/paper/2510.22049/) (2025)
- [HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing](https://mlsanity.com/paper/2602.03560/) (2026)
- [TiM4Rec: An Efficient Sequential Recommendation Model Based on Time-Aware Structured State Space Duality Model](https://mlsanity.com/paper/2409.16182/) (2024)
- [UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systems](https://mlsanity.com/paper/2604.00590/) (2026)
- [FuXi-Linear: Unleashing the Power of Linear Attention in Long-term Time-aware Sequential Recommendation](https://mlsanity.com/paper/2602.23671/) (2026)
- [UltraMemV2: Memory Networks Scaling to 120B Parameters with Superior Long-Context Learning](https://mlsanity.com/paper/2508.18756/) (2025)
- [Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials](https://mlsanity.com/paper/2511.00833/) (2025)
- [HyTRec: A Hybrid Temporal-Aware Attention Architecture for Long Behavior Sequential Recommendation](https://mlsanity.com/paper/2602.18283/) (2026)
- [SARM: LLM-Augmented Semantic Anchor for End-to-End Live-Streaming Ranking](https://mlsanity.com/paper/2602.09401/) (2026)
- [Recurrent Preference Memory for Efficient Long-Sequence Generative Recommendation](https://mlsanity.com/paper/2602.11605/) (2026)
- [HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders](https://mlsanity.com/paper/2602.21009/) (2026)
- [Dynamic User Interest Augmentation via Stream Clustering and Memory Networks in Large-Scale Recommender Systems](https://mlsanity.com/paper/2405.13238/) (2024)
- [Large Memory Network for Recommendation](https://mlsanity.com/paper/2502.05558/) (2025)
- [BlossomRec: Block-level Fused Sparse Attention Mechanism for Sequential Recommendations](https://mlsanity.com/paper/2512.13368/) (2025)

## Abstract (arXiv)

Linear attention provides an efficient backbone for long-sequence recommendation by avoiding the quadratic cost of standard Transformers, but its compressed recurrent state can be dominated by repetitive behavior patterns. We identify this phenomenon as semantic state sink, where recurring semantics over-occupy the recurrent state and bias subsequent readouts. To mitigate semantic state sink, we propose SinkRec, a hybrid memory-transition looped architecture that decouples collaborative behavioral pattern storage from dynamic transition modeling. SinkRec externalizes recurring local patterns into a learnable conditional memory through residual vector quantization, reinjects the retrieved codes, and exposes memory key-value pairs to the attention block. It further introduces Temporal-Aware State-Relation Differential Gated DeltaNet (TDGD), which uses memory to purify recurrent writing and reading by suppressing memory-covered updates and removing memory-aligned readout responses. This design turns recurring semantics from state-competing signals into memory-retrievable patterns, allowing the recurrent state to focus on dynamic transitions and alleviating semantic state sink with linear-time efficiency. Experiments on public and industrial datasets demonstrate the effectiveness and efficiency of SinkRec.