# HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders

arXiv: https://arxiv.org/abs/2602.21009 · HTML: https://mlsanity.com/paper/2602.21009/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 24 Feb 2026.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Autoregressive Image Generation using Residual Quantization](https://mlsanity.com/paper/2203.01941/) (2022)
- [Efficient Content-Based Sparse Attention with Routing Transformers](https://mlsanity.com/paper/2003.05997/) (2020)
- [Efficient Attention: Attention with Linear Complexities](https://mlsanity.com/paper/1812.01243/) (2018)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction](https://mlsanity.com/paper/2006.05639/) (2020)
- [KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems](https://mlsanity.com/paper/2202.10842/) (2022)
- [Controllable Multi-Interest Framework for Recommendation](https://mlsanity.com/paper/2005.09347/) (2020)
- [LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders](https://mlsanity.com/paper/2505.04421/) (2025)
- [Estimating the Optimal Number of Clusters in Categorical Data Clustering by Silhouette Coefficient](https://mlsanity.com/paper/2501.15542/) (2025)
- [MUSE: A Simple Yet Effective Multimodal Search-Based Framework for Lifelong User Interest Modeling](https://mlsanity.com/paper/2512.07216/) (2025)
- [Multi-Grained Patch Training for Efficient LLM-based Recommendation](https://mlsanity.com/paper/2501.15087/) (2025)
- [ELASTIC: Efficient Linear Attention for Sequential Interest Compression](https://mlsanity.com/paper/2408.09380/) (2024)

## Built on by (most cited first, 3 of 3)

- [AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems](https://mlsanity.com/paper/2606.26859/) (2026)
- [SinkRec: Mitigating Semantic State Sink in Long Sequence Recommendation with Memory-Conditioned Gated Delta Networks](https://mlsanity.com/paper/2606.09888/) (2026)
- [Tokens are All You Need: Dual-purpose Semantic IDs for Achieving LLM-Level I/O Efficiency in recommendation systems](https://mlsanity.com/paper/2607.24865/) (2026)

## Abstract (arXiv)

Modern recommender systems leverage ultra-long user behavior sequences to capture dynamic preferences, but end-to-end modeling is infeasible in production due to latency and memory constraints. While summarizing history via interest centers offers a practical alternative, existing methods struggle to (1) identify user-specific centers at appropriate granularity and (2) accurately assign behaviors, leading to quantization errors and loss of long-tail preferences. To alleviate these issues, we propose Hierarchical Sparse Activation Compression (HiSAC), an efficient framework for personalized sequence modeling. HiSAC encodes interactions into multi-level semantic IDs and constructs a global hierarchical codebook. A hierarchical voting mechanism sparsely activates personalized interest-agents as fine-grained preference centers. Guided by these agents, Soft-Routing Attention aggregates historical signals in semantic space, weighting by similarity to minimize quantization error and retain long-tail behaviors. Deployed on Taobao's "Guess What You Like" homepage, HiSAC achieves significant compression and cost reduction, with online A/B tests showing a consistent 1.65% CTR uplift -- demonstrating its scalability and real-world effectiveness.