# UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systems

arXiv: https://arxiv.org/abs/2604.00590 · HTML: https://mlsanity.com/paper/2604.00590/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 28 references that are in the map.
- First posted on arXiv 1 Apr 2026.

## Builds on (most cited first, 22 of 22)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [Wukong: Towards a Scaling Law for Large-Scale Recommendation](https://mlsanity.com/paper/2403.02545/) (2024)
- [RankMixer: Scaling Up Ranking Models in Industrial Recommenders](https://mlsanity.com/paper/2507.15551/) (2025)
- [OneTrans: Unified Feature Interaction and Sequence Modeling with One Transformer in Industrial Recommender](https://mlsanity.com/paper/2510.26104/) (2025)
- [Hiformer: Heterogeneous Feature Interactions Learning with Transformers for Recommender Systems](https://mlsanity.com/paper/2311.05884/) (2023)
- [SiameseNorm: Breaking the Barrier to Reconciling Pre/Post-Norm](https://mlsanity.com/paper/2602.08064/) (2026)
- [Climber: Toward Efficient Scaling Laws for Large Recommendation Models](https://mlsanity.com/paper/2502.09888/) (2025)
- [Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design](https://mlsanity.com/paper/2602.10016/) (2026)
- [HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction](https://mlsanity.com/paper/2601.12681/) (2026)
- [LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation](https://mlsanity.com/paper/2601.20083/) (2026)
- [LEMUR: Large scale End-to-end MUltimodal Recommendation](https://mlsanity.com/paper/2511.10962/) (2025)
- [HHFT: Hierarchical Heterogeneous Feature Transformer for Recommendation Systems](https://mlsanity.com/paper/2511.20235/) (2025)
- [TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders](https://mlsanity.com/paper/2602.06563/) (2026)
- [Bending the Scaling Law Curve in Large-Scale Recommendation Systems](https://mlsanity.com/paper/2602.16986/) (2026)
- [MTmixAtt: Integrating Mixture-of-Experts with Multi-Mix Attention for Large-Scale Recommendation](https://mlsanity.com/paper/2510.15286/) (2025)
- [From Scaling to Structured Expressivity: Rethinking Transformers for CTR Prediction](https://mlsanity.com/paper/2511.12081/) (2025)

## Built on by (most cited first, 7 of 7)

- [Field-Aware RankMixer with Dual-Stream Bilinear Fusion for the Tencent UNI-REC Challenge](https://mlsanity.com/paper/2607.15590/) (2026)
- [UniDot: A Unified Network for Sequence Modeling and Feature Interaction in Large-scale Recommendation](https://mlsanity.com/paper/2608.16797/) (2026)
- [SinkRec: Mitigating Semantic State Sink in Long Sequence Recommendation with Memory-Conditioned Gated Delta Networks](https://mlsanity.com/paper/2606.09888/) (2026)
- [WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture](https://mlsanity.com/paper/2607.17017/) (2026)
- [RoleMix: Unifying Sequential and Non-Sequential Features via Semantic Tokenization for Post-Click Conversion Rate Prediction](https://mlsanity.com/paper/2607.22700/) (2026)
- [QueryFormer: Winning Solution for KDD Cup 2026 Tencent UniRec Challenge](https://mlsanity.com/paper/2609.16548/) (2026)
- [Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems](https://mlsanity.com/paper/2609.30001/) (2026)

## Abstract (arXiv)

In recent years, the scaling laws of recommendation models have attracted increasing attention, which govern the relationship between performance and parameters/FLOPs of recommenders. Currently, there are three mainstream architectures for achieving scaling in recommendation models, namely attention-based, TokenMixer-based, and factorization-machine-based methods, which exhibit fundamental differences in both design philosophy and architectural structure. In this paper, we propose a unified scaling architecture for recommendation systems, namely UniMixer, to improve scaling efficiency and establish a unified theoretical framework that unifies the mainstream scaling blocks. By transforming the rule-based TokenMixer to an equivalent parameterized structure, we construct a generalized parameterized feature mixing module that allows the token mixing patterns to be optimized and learned during model training. Meanwhile, the generalized parameterized token mixing removes the constraint in TokenMixer that requires the number of heads to be equal to the number of tokens. Furthermore, we establish a unified scaling module design framework for recommender systems, which bridges the connections among attention-based, TokenMixer-based, and factorization-machine-based methods. To further boost scaling ROI, a lightweight UniMixing module is designed, UniMixing-Lite, which further compresses the model parameters and computational cost while significantly improve the model performance. The scaling curves are shown in the following figure. Extensive offline and online experiments are conducted to verify the superior scaling abilities of UniMixer.