# LREA: Low-Rank Efficient Attention on Modeling Long-Term User Behaviors for CTR Prediction

arXiv: https://arxiv.org/abs/2503.02542 · HTML: https://mlsanity.com/paper/2503.02542/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2025.

## Built on by (most cited first, 3 of 3)

- [DiffoR: A Unified Continuous Generative Framework for Universal Ordinal Regression](https://mlsanity.com/paper/2606.07599/) (2026)
- [LIME: Link-based user-item Interaction Modeling with decoupled xor attention for Efficient test time scaling](https://mlsanity.com/paper/2510.18239/) (2025)
- [SOLAR: SVD-Optimized Lifelong Attention for Recommendation](https://mlsanity.com/paper/2603.02561/) (2026)

## Abstract (arXiv)

With the rapid growth of user historical behavior data, user interest modeling has become a prominent aspect in Click-Through Rate (CTR) prediction, focusing on learning user intent representations. However, this complexity poses computational challenges, requiring a balance between model performance and acceptable response times for online services. Traditional methods often utilize filtering techniques. These techniques can lead to the loss of significant information by prioritizing top K items based on item attributes or employing low-precision attention mechanisms. In this study, we introduce LREA, a novel attention mechanism that overcomes the limitations of existing approaches while ensuring computational efficiency. LREA leverages low-rank matrix decomposition to optimize runtime performance and incorporates a specially designed loss function to maintain attention capabilities while preserving information integrity. During the inference phase, matrix absorption and pre-storage strategies are employed to effectively meet runtime constraints. The results of extensive offline and online experiments demonstrate that our method outperforms state-of-the-art approaches.