# LambdaOpt: Learn to Regularize Recommender Models in Finer Levels

arXiv: https://arxiv.org/abs/1905.11596 · HTML: https://mlsanity.com/paper/1905.11596/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2019.

## Built on by (most cited first, 14 of 14)

- [AutoDebias: Learning to Debias for Recommendation](https://mlsanity.com/paper/2105.04170/) (2021)
- [Learning with Multiclass AUC: Theory and Algorithms](https://mlsanity.com/paper/2107.13171/) (2021)
- [Gradient-based Bi-level Optimization for Deep Learning: A Survey](https://mlsanity.com/paper/2207.11719/) (2022)
- [Scaling Embedding Layers in Language Models](https://mlsanity.com/paper/2502.01637/) (2025)
- [Click-through Rate Prediction with Auto-Quantized Contrastive Learning](https://mlsanity.com/paper/2109.13921/) (2021)
- [A Comprehensive Survey on Automated Machine Learning for Recommendations](https://mlsanity.com/paper/2204.01390/) (2022)
- [FreezeAsGuard: Mitigating Illegal Adaptation of Diffusion Models via Selective Tensor Freezing](https://mlsanity.com/paper/2405.17472/) (2024)
- [Future Gradient Descent for Adapting the Temporal Shifting Data Distribution in Online Recommendation Systems](https://mlsanity.com/paper/2209.01143/) (2022)
- [Adapting Triplet Importance of Implicit Feedback for Personalized Recommendation](https://mlsanity.com/paper/2208.01709/) (2022)
- [AutoAL: Automated Active Learning with Differentiable Query Strategy Search](https://mlsanity.com/paper/2410.13853/) (2024)
- [Efficient Click-Through Rate Prediction for Developing Countries via Tabular Learning](https://mlsanity.com/paper/2104.07553/) (2021)
- [Learning Robust Recommender from Noisy Implicit Feedback](https://mlsanity.com/paper/2112.01160/) (2021)
- [Adaptive Latent Factor Analysis via Generalized Momentum-Incorporated Particle Swarm Optimization](https://mlsanity.com/paper/2208.02423/) (2022)
- [Automatic Self-supervised Learning for Social Recommendations](https://mlsanity.com/paper/2412.18735/) (2024)

## Abstract (arXiv)

Recommendation models mainly deal with categorical variables, such as user/item ID and attributes. Besides the high-cardinality issue, the interactions among such categorical variables are usually long-tailed, with the head made up of highly frequent values and a long tail of rare ones. This phenomenon results in the data sparsity issue, making it essential to regularize the models to ensure generalization. The common practice is to employ grid search to manually tune regularization hyperparameters based on the validation data. However, it requires non-trivial efforts and large computation resources to search the whole candidate space; even so, it may not lead to the optimal choice, for which different parameters should have different regularization strengths. In this paper, we propose a hyperparameter optimization method, LambdaOpt, which automatically and adaptively enforces regularization during training. Specifically, it updates the regularization coefficients based on the performance of validation data. With LambdaOpt, the notorious tuning of regularization hyperparameters can be avoided; more importantly, it allows fine-grained regularization (i.e. each parameter can have an individualized regularization coefficient), leading to better generalized models. We show how to employ LambdaOpt on matrix factorization, a classical model that is representative of a large family of recommender models. Extensive experiments on two public benchmarks demonstrate the superiority of our method in boosting the performance of top-K recommendation.