# TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders

arXiv: https://arxiv.org/abs/2602.06563 · HTML: https://mlsanity.com/paper/2602.06563/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in MoE for recommender systems, one of 12 directions in Mixture-of-experts.
- First posted on arXiv 6 Feb 2026.

## Placed in

- Mixture-of-experts / [MoE for recommender systems](https://mlsanity.com/moe/moe-for-recommender-systems/)

## Built on by (most cited first, 4 of 4)

- [UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systems](https://mlsanity.com/paper/2604.00590/) (2026)
- [Taiji: Pareto Optimal Policy Optimization with Semantics-IDs Trade-off for Industrial LLM-Enhanced Recommendation](https://mlsanity.com/paper/2606.03866/) (2026)
- [UniMoMo: Expert Merging-Based MoE Acceleration for Large Recommendation Models](https://mlsanity.com/paper/2608.08627/) (2026)
- [Token Factory: Efficiently Integrating Diverse Signals into Large Recommendation Models](https://mlsanity.com/paper/2606.19635/) (2026)

## Abstract (arXiv)

While scaling laws for recommendation models have gained significant traction, existing architectures such as Wukong, HiFormer and DHEN, often struggle with sub-optimal designs and hardware under-utilization, limiting their practical scalability. Our previous TokenMixer architecture (introduced in RankMixer paper) addressed effectiveness and efficiency by replacing self-attention with a ightweight token-mixing operator; however, it faced critical bottlenecks in deeper configurations, including sub-optimal residual paths, vanishing gradients, incomplete MoE sparsification and constrained scalability. In this paper, we propose TokenMixer-Large, a systematically evolved architecture designed for extreme-scale recommendation. By introducing a mixing-and-reverting operation, inter-layer residuals and the auxiliary loss, we ensure stable gradient propagation even as model depth increases. Furthermore, we incorporate a Sparse Per-token MoE to enable efficient parameter expansion. TokenMixer-Large successfully scales its parameters to 7-billion and 15-billion on online traffic and offline experiments, respectively. Currently deployed in multiple scenarios at ByteDance, TokenMixer-Large has achieved significant offline and online performance gains, delivering an increase of +1.66% in orders and +2.98% in per-capita preview payment GMV for e-commerce, improving ADSS by +2.0% in advertising and achieving a +1.4% revenue growth for live streaming.