# AutoLoss: Automated Loss Function Search in Recommendations

arXiv: https://arxiv.org/abs/2106.06713 · HTML: https://mlsanity.com/paper/2106.06713/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 68 references that are in the map.
- First posted on arXiv 12 Jun 2021.

## Builds on (most cited first, 30 of 43)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [Large-Margin Softmax Loss for Convolutional Neural Networks](https://mlsanity.com/paper/1612.02295/) (2016)
- [xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems](https://mlsanity.com/paper/1803.05170/) (2018)
- [Neural Factorization Machines for Sparse Predictive Analytics](https://mlsanity.com/paper/1708.05027/) (2017)
- [UPSNet: A Unified Panoptic Segmentation Network](https://mlsanity.com/paper/1901.03784/) (2019)
- [Product-based Neural Networks for User Response Prediction](https://mlsanity.com/paper/1611.00144/) (2016)
- [MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendation](https://mlsanity.com/paper/1908.00413/) (2019)
- [Deep Reinforcement Learning for Page-wise Recommendations](https://mlsanity.com/paper/1805.02343/) (2018)
- [Improved Recurrent Neural Networks for Session-based Recommendations](https://mlsanity.com/paper/1606.08117/) (2016)
- [Towards Long-term Fairness in Recommendation](https://mlsanity.com/paper/2101.03584/) (2021)
- [Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning](https://mlsanity.com/paper/1802.06501/) (2018)
- [AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction](https://mlsanity.com/paper/2003.11235/) (2020)
- [Automated Self-Supervised Learning for Graphs](https://mlsanity.com/paper/2106.05470/) (2021)
- [AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations](https://mlsanity.com/paper/2002.11252/) (2020)
- [DEAR: Deep Reinforcement Learning for Online Advertising Impression in Recommender Systems](https://mlsanity.com/paper/1909.03602/) (2019)
- [Learnable Embedding Sizes for Recommender Systems](https://mlsanity.com/paper/2101.07577/) (2021)
- [Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems](https://mlsanity.com/paper/1909.11810/) (2019)
- [AutoCross: Automatic Feature Crossing for Tabular Data in Real-World Applications](https://mlsanity.com/paper/1904.12857/) (2019)
- [Neural Input Search for Large Scale Recommendation Models](https://mlsanity.com/paper/1907.04471/) (2019)
- [Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction](https://mlsanity.com/paper/2007.06434/) (2020)
- [Security and Privacy on Blockchain](https://mlsanity.com/paper/1903.07602/) (2019)
- [Feature Interaction Interpretability: A Case for Explaining Ad-Recommendation Systems via Neural Interaction Detection](https://mlsanity.com/paper/2006.10966/) (2020)
- [Attacking Black-box Recommendations via Copying Cross-domain User Profiles](https://mlsanity.com/paper/2005.08147/) (2020)

## Built on by (most cited first, 30 of 31)

- [Recommender Systems in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2307.02046/) (2023)
- [RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs](https://mlsanity.com/paper/2503.10657/) (2025)
- [Graph Trend Filtering Networks for Recommendations](https://mlsanity.com/paper/2108.05552/) (2021)
- [AutoField: Automating Feature Selection in Deep Recommender Systems](https://mlsanity.com/paper/2204.09078/) (2022)
- [IMF: Interactive Multimodal Fusion Model for Link Prediction](https://mlsanity.com/paper/2303.10816/) (2023)
- [Disentangled Contrastive Learning for Social Recommendation](https://mlsanity.com/paper/2208.08723/) (2022)
- [Knowledge-enhanced Black-box Attacks for Recommendations](https://mlsanity.com/paper/2207.10307/) (2022)
- [AutoMLP: Automated MLP for Sequential Recommendations](https://mlsanity.com/paper/2303.06337/) (2023)
- [Jointly Attacking Graph Neural Network and its Explanations](https://mlsanity.com/paper/2108.03388/) (2021)
- [Memorize, Factorize, or be Na"ive: Learning Optimal Feature Interaction Methods for CTR Prediction](https://mlsanity.com/paper/2108.01265/) (2021)
- [AutoLossGen: Automatic Loss Function Generation for Recommender Systems](https://mlsanity.com/paper/2204.13160/) (2022)
- [AutoSTL: Automated Spatio-Temporal Multi-Task Learning](https://mlsanity.com/paper/2304.09174/) (2023)
- [ERASE: Benchmarking Feature Selection Methods for Deep Recommender Systems](https://mlsanity.com/paper/2403.12660/) (2024)
- [PSL: Rethinking and Improving Softmax Loss from Pairwise Perspective for Recommendation](https://mlsanity.com/paper/2411.00163/) (2024)
- [Toward a Better Understanding of Loss Functions for Collaborative Filtering](https://mlsanity.com/paper/2308.06091/) (2023)
- [OptDist: Learning Optimal Distribution for Customer Lifetime Value Prediction](https://mlsanity.com/paper/2408.08585/) (2024)
- [Automated Graph Self-supervised Learning via Multi-teacher Knowledge Distillation](https://mlsanity.com/paper/2210.02099/) (2022)
- [Consensus Learning from Heterogeneous Objectives for One-Class Collaborative Filtering](https://mlsanity.com/paper/2202.13140/) (2022)
- [BSL: Understanding and Improving Softmax Loss for Recommendation](https://mlsanity.com/paper/2312.12882/) (2023)
- [Automated Contrastive Learning Strategy Search for Time Series](https://mlsanity.com/paper/2403.12641/) (2024)
- [A Comprehensive Survey on Automated Machine Learning for Recommendations](https://mlsanity.com/paper/2204.01390/) (2022)
- [ColdNAS: Search to Modulate for User Cold-Start Recommendation](https://mlsanity.com/paper/2306.03387/) (2023)
- [DNS-Rec: Data-aware Neural Architecture Search for Recommender Systems](https://mlsanity.com/paper/2402.00390/) (2024)
- [RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems](https://mlsanity.com/paper/2607.29241/) (2026)
- [DANCE: Resource-Efficient Neural Architecture Search with Data-Aware and Continuous Adaptation](https://mlsanity.com/paper/2507.04671/) (2025)
- [Prompt-Based LLMs for Position Bias-Aware Reranking in Personalized Recommendations](https://mlsanity.com/paper/2505.04948/) (2025)
- [Predicting Disease Progress with Imprecise Lab Test Results](https://mlsanity.com/paper/2107.03620/) (2021)
- [SPR:Supervised Personalized Ranking Based on Prior Knowledge for Recommendation](https://mlsanity.com/paper/2207.03197/) (2022)
- [Multi-Margin Cosine Loss: Proposal and Application in Recommender Systems](https://mlsanity.com/paper/2405.04614/) (2024)
- [Renormalization Group Guided Tensor Network Structure Search](https://mlsanity.com/paper/2512.24663/) (2025)

## Abstract (arXiv)

Designing an effective loss function plays a crucial role in training deep recommender systems. Most existing works often leverage a predefined and fixed loss function that could lead to suboptimal recommendation quality and training efficiency. Some recent efforts rely on exhaustively or manually searched weights to fuse a group of candidate loss functions, which is exceptionally costly in computation and time. They also neglect the various convergence behaviors of different data examples. In this work, we propose an AutoLoss framework that can automatically and adaptively search for the appropriate loss function from a set of candidates. To be specific, we develop a novel controller network, which can dynamically adjust the loss probabilities in a differentiable manner. Unlike existing algorithms, the proposed controller can adaptively generate the loss probabilities for different data examples according to their varied convergence behaviors. Such design improves the model's generalizability and transferability between deep recommender systems and datasets. We evaluate the proposed framework on two benchmark datasets. The results show that AutoLoss outperforms representative baselines. Further experiments have been conducted to deepen our understandings of AutoLoss, including its transferability, components and training efficiency.