# FuXi-β: Towards a Lightweight and Fast Large-Scale Generative Recommendation Model

arXiv: https://arxiv.org/abs/2508.10615 · HTML: https://mlsanity.com/paper/2508.10615/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Aug 2025.

## Built on by (most cited first, 6 of 6)

- [Why Thinking Hurts: Diagnosing and Rectifying Linguistic Inertia in Large Language Models for Recommendation](https://mlsanity.com/paper/2602.16587/) (2026)
- [RAG-IGBench: Innovative Evaluation for RAG-based Interleaved Generation in Open-domain Question Answering](https://mlsanity.com/paper/2512.05119/) (2025)
- [Revisiting scalable sequential recommendation with Multi-Embedding Approach and Mixture-of-Experts](https://mlsanity.com/paper/2510.25285/) (2025)
- [Efficient Personalized Reranking with Semi-Autoregressive Generation and Online Knowledge Distillation](https://mlsanity.com/paper/2603.07107/) (2026)
- [UniPinRec: Unifying Generative Retrieval and Ranking at Pinterest Scale](https://mlsanity.com/paper/2606.00422/) (2026)
- [Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems](https://mlsanity.com/paper/2609.30001/) (2026)

## Abstract (arXiv)

Scaling laws for autoregressive generative recommenders reveal potential for larger, more versatile systems but mean greater latency and training costs. To accelerate training and inference, we investigated the recent generative recommendation models HSTU and FuXi-α, identifying two efficiency bottlenecks: the indexing operations in relative temporal attention bias and the computation of the query-key attention map. Additionally, we observed that relative attention bias in self-attention mechanisms can also serve as attention maps. Previous works like Synthesizer have shown that alternative forms of attention maps can achieve similar performance, naturally raising the question of whether some attention maps are redundant. Through empirical experiments, we discovered that using the query-key attention map might degrade the model's performance in recommendation tasks. To address these bottlenecks, we propose a new framework applicable to Transformer-like recommendation models. On one hand, we introduce Functional Relative Attention Bias, which avoids the time-consuming operations of the original relative attention bias, thereby accelerating the process. On the other hand, we remove the query-key attention map from the original self-attention layer and design a new Attention-Free Token Mixer module. Furthermore, by applying this framework to FuXi-α, we introduce a new model, FuXi-β. Experiments across multiple datasets demonstrate that FuXi-β outperforms previous state-of-the-art models and achieves significant acceleration compared to FuXi-α, while also adhering to the scaling law. Notably, FuXi-β shows an improvement of 27% to 47% in the NDCG@10 metric on large-scale industrial datasets compared to FuXi-α. Our code is available in a public repository: https://github.com/USTC-StarTeam/FuXi-beta