# AutoML for Large Capacity Modeling of Meta's Ranking Systems

arXiv: https://arxiv.org/abs/2311.07870 · HTML: https://mlsanity.com/paper/2311.07870/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 35 references that are in the map.
- First posted on arXiv 14 Nov 2023.

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [AutoML: A Survey of the State-of-the-Art](https://mlsanity.com/paper/1908.00709/) (2019)
- [Large-Scale Evolution of Image Classifiers](https://mlsanity.com/paper/1703.01041/) (2017)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems](https://mlsanity.com/paper/1803.05170/) (2018)
- [A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions](https://mlsanity.com/paper/2006.02903/) (2020)
- [Neural Architecture Search with Bayesian Optimisation and Optimal Transport](https://mlsanity.com/paper/1802.07191/) (2018)
- [Neural Factorization Machines for Sparse Predictive Analytics](https://mlsanity.com/paper/1708.05027/) (2017)
- [Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution](https://mlsanity.com/paper/1804.09081/) (2018)
- [Neural Predictor for Neural Architecture Search](https://mlsanity.com/paper/1912.00848/) (2019)
- [Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models](https://mlsanity.com/paper/2104.05158/) (2021)
- [A Survey on Neural Architecture Search](https://mlsanity.com/paper/1905.01392/) (2019)
- [A Genetic Programming Approach to Designing Convolutional Neural Network Architectures](https://mlsanity.com/paper/1704.00764/) (2017)
- [AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction](https://mlsanity.com/paper/2003.11235/) (2020)
- [Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search](https://mlsanity.com/paper/1807.06906/) (2018)
- [Hyperparameter Search in Machine Learning](https://mlsanity.com/paper/1502.02127/) (2015)
- [Automated Machine Learning: State-of-The-Art and Open Challenges](https://mlsanity.com/paper/1906.02287/) (2019)
- [Benchmark and Survey of Automated Machine Learning Frameworks](https://mlsanity.com/paper/1904.12054/) (2019)
- [Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spaces](https://mlsanity.com/paper/1409.4011/) (2014)

## Built on by (most cited first, 4 of 4)

- [Edge-Cloud Collaborative Computing on Distributed Intelligence and Model Optimization: A Survey](https://mlsanity.com/paper/2505.01821/) (2025)
- [CubicML: Automated ML for Large ML Systems Co-design with ML Prediction of Performance](https://mlsanity.com/paper/2409.04585/) (2024)
- [Design Once, Deploy at Scale: Template-Driven ML Development for Large Model Ecosystems](https://mlsanity.com/paper/2603.24963/) (2026)
- [Bi-NAS: Towards Effective and Personalized Explanation for Recommender Systems via Bi-Level Neural Architecture Search](https://mlsanity.com/paper/2607.01387/) (2026)

## Abstract (arXiv)

Web-scale ranking systems at Meta serving billions of users is complex. Improving ranking models is essential but engineering heavy. Automated Machine Learning (AutoML) can release engineers from labor intensive work of tuning ranking models; however, it is unknown if AutoML is efficient enough to meet tight production timeline in real-world and, at the same time, bring additional improvements to the strong baselines. Moreover, to achieve higher ranking performance, there is an ever-increasing demand to scale up ranking models to even larger capacity, which imposes more challenges on the efficiency. The large scale of models and tight production schedule requires AutoML to outperform human baselines by only using a small number of model evaluation trials (around 100). We presents a sampling-based AutoML method, focusing on neural architecture search and hyperparameter optimization, addressing these challenges in Meta-scale production when building large capacity models. Our approach efficiently handles large-scale data demands. It leverages a lightweight predictor-based searcher and reinforcement learning to explore vast search spaces, significantly reducing the number of model evaluations. Through experiments in large capacity modeling for CTR and CVR applications, we show that our method achieves outstanding Return on Investment (ROI) versus human tuned baselines, with up to 0.09% Normalized Entropy (NE) loss reduction or 25% Query per Second (QPS) increase by only sampling one hundred models on average from a curated search space. The proposed AutoML method has already made real-world impact where a discovered Instagram CTR model with up to -0.36% NE gain (over existing production baseline) was selected for large-scale online A/B test and show statistically significant gain. These production results proved AutoML efficacy and accelerated its adoption in ranking systems at Meta.