# RankMixer: Scaling Up Ranking Models in Industrial Recommenders

arXiv: https://arxiv.org/abs/2507.15551 · HTML: https://mlsanity.com/paper/2507.15551/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2025.

## Built on by (most cited first, 30 of 42)

- [MiniOneRec: An Open-Source Framework for Scaling Generative Recommendation](https://mlsanity.com/paper/2510.24431/) (2025)
- [Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design](https://mlsanity.com/paper/2602.10016/) (2026)
- [AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems](https://mlsanity.com/paper/2606.26859/) (2026)
- [UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systems](https://mlsanity.com/paper/2604.00590/) (2026)
- [NEZHA: A Zero-sacrifice and Hyperspeed Decoding Architecture for Generative Recommendations](https://mlsanity.com/paper/2511.18793/) (2025)
- [MTmixAtt: Integrating Mixture-of-Experts with Multi-Mix Attention for Large-Scale Recommendation](https://mlsanity.com/paper/2510.15286/) (2025)
- [From Scaling to Structured Expressivity: Rethinking Transformers for CTR Prediction](https://mlsanity.com/paper/2511.12081/) (2025)
- [RelayGR: Scaling Long-Sequence Generative Recommendation via Cross-Stage Relay-Race Inference](https://mlsanity.com/paper/2601.01712/) (2026)
- [MuonRec: Shifting the Optimizer Paradigm Beyond Adam in Scalable Generative Recommendation](https://mlsanity.com/paper/2603.00416/) (2026)
- [Taiji: Pareto Optimal Policy Optimization with Semantics-IDs Trade-off for Industrial LLM-Enhanced Recommendation](https://mlsanity.com/paper/2606.03866/) (2026)
- [UniMoMo: Expert Merging-Based MoE Acceleration for Large Recommendation Models](https://mlsanity.com/paper/2608.08627/) (2026)
- [ERASE: EaRly bAckpropagation SchEdule for Faster Training of Modern Recommendation Systems](https://mlsanity.com/paper/2608.18469/) (2026)
- [Target-Aware Early Stage Ranking](https://mlsanity.com/paper/2511.21095/) (2025)
- [Compute Only Once: UG-Separation for Efficient Large Recommendation Models](https://mlsanity.com/paper/2602.10455/) (2026)
- [MAC: A Conversion Rate Prediction Benchmark Featuring Labels Under Multiple Attribution Mechanisms](https://mlsanity.com/paper/2603.02184/) (2026)
- [Closing the Auto-Research Loop: An AI Co-Scientist for Production Search Ranking](https://mlsanity.com/paper/2603.22376/) (2026)
- [Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendation](https://mlsanity.com/paper/2605.23191/) (2026)
- [Token Factory: Efficiently Integrating Diverse Signals into Large Recommendation Models](https://mlsanity.com/paper/2606.19635/) (2026)
- [SlimPer: Make Personalization Model Slim and Smart](https://mlsanity.com/paper/2607.12281/) (2026)
- [Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning](https://mlsanity.com/paper/2607.14192/) (2026)
- [Field-Aware RankMixer with Dual-Stream Bilinear Fusion for the Tencent UNI-REC Challenge](https://mlsanity.com/paper/2607.15590/) (2026)
- [From Trajectories to Evidence: Auditable Experimental Records for Industrial Research Agents](https://mlsanity.com/paper/2608.05235/) (2026)
- [MATT-CTR: Unleashing a Model-Agnostic Test-Time Paradigm for CTR Prediction with Confidence-Guided Inference Paths](https://mlsanity.com/paper/2510.08932/) (2025)
- [Make It Long, Keep It Fast: End-to-End 10K Long User Behavior Sequence Modeling for Billion-Scale Douyin Recommendation](https://mlsanity.com/paper/2511.06077/) (2025)
- [Exploring Test-time Scaling via Prediction Merging on Large-Scale Recommendation](https://mlsanity.com/paper/2512.07650/) (2025)
- [MDL: A Unified Multi-Distribution Learner in Large-scale Industrial Recommendation through Tokenization](https://mlsanity.com/paper/2602.07520/) (2026)
- [ML-DCN: Masked Low-Rank Deep Crossing Network Towards Scalable Ads Click-through Rate Prediction at Pinterest](https://mlsanity.com/paper/2602.09194/) (2026)
- [Deferred is Better: A Framework for Multi-Granularity Deferred Interaction of Heterogeneous Features](https://mlsanity.com/paper/2603.12586/) (2026)
- [Selective Test-Time Compute Scaling for Click-Through Rate Prediction via Uncertainty-Triggered Feature Path Exploration](https://mlsanity.com/paper/2605.24989/) (2026)
- [LeAP: Learnable Adaptive Permutation for Feature Selection in Heterogeneous and Sparse Recommender Systems](https://mlsanity.com/paper/2606.01111/) (2026)

## Abstract (arXiv)

Recent progress on large language models (LLMs) has spurred interest in scaling up recommendation systems, yet two practical obstacles remain. First, training and serving cost on industrial Recommenders must respect strict latency bounds and high QPS demands. Second, most human-designed feature-crossing modules in ranking models were inherited from the CPU era and fail to exploit modern GPUs, resulting in low Model Flops Utilization (MFU) and poor scalability. We introduce RankMixer, a hardware-aware model design tailored towards a unified and scalable feature-interaction architecture. RankMixer retains the transformer's high parallelism while replacing quadratic self-attention with multi-head token mixing module for higher efficiency. Besides, RankMixer maintains both the modeling for distinct feature subspaces and cross-feature-space interactions with Per-token FFNs. We further extend it to one billion parameters with a Sparse-MoE variant for higher ROI. A dynamic routing strategy is adapted to address the inadequacy and imbalance of experts training. Experiments show RankMixer's superior scaling abilities on a trillion-scale production dataset. By replacing previously diverse handcrafted low-MFU modules with RankMixer, we boost the model MFU from 4.5% to 45%, and scale our ranking model parameters by 100x while maintaining roughly the same inference latency. We verify RankMixer's universality with online A/B tests across two core application scenarios (Recommendation and Advertisement). Finally, we launch 1B Dense-Parameters RankMixer for full traffic serving without increasing the serving cost, which improves user active days by 0.3% and total in-app usage duration by 1.08%.