# Advancing Loss Functions in Recommender Systems: A Comparative Study with a R'enyi Divergence-Based Solution

arXiv: https://arxiv.org/abs/2506.15120 · HTML: https://mlsanity.com/paper/2506.15120/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 67 references that are in the map.
- First posted on arXiv 18 Jun 2025.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [Graph Neural Networks in Recommender Systems: A Survey](https://mlsanity.com/paper/2011.02260/) (2020)
- [Learning Models with Uniform Performance via Distributionally Robust Optimization](https://mlsanity.com/paper/1810.08750/) (2018)
- [Self-supervised Graph Learning for Recommendation](https://mlsanity.com/paper/2010.10783/) (2020)
- [Bias and Debias in Recommender System: A Survey and Future Directions](https://mlsanity.com/paper/2010.03240/) (2020)
- [VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback](https://mlsanity.com/paper/1510.01784/) (2015)
- [AutoDebias: Learning to Debias for Recommendation](https://mlsanity.com/paper/2105.04170/) (2021)
- [SimpleX: A Simple and Strong Baseline for Collaborative Filtering](https://mlsanity.com/paper/2109.12613/) (2021)
- [Towards Representation Alignment and Uniformity in Collaborative Filtering](https://mlsanity.com/paper/2206.12811/) (2022)
- [XSimGCL: Towards Extremely Simple Graph Contrastive Learning for Recommendation](https://mlsanity.com/paper/2209.02544/) (2022)
- [DORO: Distributional and Outlier Robust Optimization](https://mlsanity.com/paper/2106.06142/) (2021)
- [On the Equivalence of Decoupled Graph Convolution Network and Label Propagation](https://mlsanity.com/paper/2010.12408/) (2020)
- [Macro Graph Neural Networks for Online Billion-Scale Recommender Systems](https://mlsanity.com/paper/2401.14939/) (2024)
- [Latent Relational Metric Learning via Memory-based Attention for Collaborative Ranking](https://mlsanity.com/paper/1707.05176/) (2017)
- [On the Effectiveness of Sampled Softmax Loss for Item Recommendation](https://mlsanity.com/paper/2201.02327/) (2022)
- [CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender System](https://mlsanity.com/paper/2204.01266/) (2022)
- [Alleviating Matthew Effect of Offline Reinforcement Learning in Interactive Recommendation](https://mlsanity.com/paper/2307.04571/) (2023)
- [Graph Convolutional Network for Recommendation with Low-pass Collaborative Filters](https://mlsanity.com/paper/2006.15516/) (2020)
- [Incorporating Bias-aware Margins into Contrastive Loss for Collaborative Filtering](https://mlsanity.com/paper/2210.11054/) (2022)
- [Self-Guided Learning to Denoise for Robust Recommendation](https://mlsanity.com/paper/2204.06832/) (2022)
- [Empowering Collaborative Filtering with Principled Adversarial Contrastive Loss](https://mlsanity.com/paper/2310.18700/) (2023)
- [Invariant Collaborative Filtering to Popularity Distribution Shift](https://mlsanity.com/paper/2302.05328/) (2023)
- [AutoDenoise: Automatic Data Instance Denoising for Recommendations](https://mlsanity.com/paper/2303.06611/) (2023)
- [Efficient Bi-Level Optimization for Recommendation Denoising](https://mlsanity.com/paper/2210.10321/) (2022)

## Built on by (most cited first, 2 of 2)

- [Talos: Optimizing Top-K Accuracy in Recommender Systems](https://mlsanity.com/paper/2601.19276/) (2026)
- [BEAR: Towards Beam-Search-Aware Optimization for Recommendation with Large Language Models](https://mlsanity.com/paper/2601.22925/) (2026)

## Abstract (arXiv)

Loss functions play a pivotal role in optimizing recommendation models. Among various loss functions, Softmax Loss (SL) and Cosine Contrastive Loss (CCL) are particularly effective. Their theoretical connections and differences warrant in-depth exploration. This work conducts comprehensive analyses of these losses, yielding significant insights: 1) Common strengths -- both can be viewed as augmentations of traditional losses with Distributional Robust Optimization (DRO), enhancing robustness to distributional shifts; 2) Respective limitations -- stemming from their use of different distribution distance metrics in DRO optimization, SL exhibits high sensitivity to false negative instances, whereas CCL suffers from low data utilization. To address these limitations, this work proposes a new loss function, DrRL, which generalizes SL and CCL by leveraging R'enyi-divergence in DRO optimization. DrRL incorporates the advantageous structures of both SL and CCL, and can be demonstrated to effectively mitigate their limitations. Extensive experiments have been conducted to validate the superiority of DrRL on both recommendation accuracy and robustness.