mlsanityData to 14 Oct 2026

PSL: Rethinking and Improving Softmax Loss from Pairwise Perspective for Recommendation

arXiv 2411.00163 ↗First posted 31 Oct 2024cs.LG cs.AI cs.IRcited by 9

Not on the map. Why?

Abstract (arXiv)

Softmax Loss (SL) is widely applied in recommender systems (RS) and has demonstrated effectiveness. This work analyzes SL from a pairwise perspective, revealing two significant limitations: 1) the relationship between SL and conventional ranking metrics like DCG is not sufficiently tight; 2) SL is highly sensitive to false negative instances. Our analysis indicates that these limitations are primarily due to the use of the exponential function. To address these issues, this work extends SL to a new family of loss functions, termed Pairwise Softmax Loss (PSL), which replaces the exponential function in SL with other appropriate activation functions. While the revision is minimal, we highlight three merits of PSL: 1) it serves as a tighter surrogate for DCG with suitable activation functions; 2) it better balances data contributions; and 3) it acts as a specific BPR loss enhanced by Distributionally Robust Optimization (DRO). We further validate the effectiveness and robustness of PSL through empirical experiments. The code is available at https://github.com/Tiny-Snow/IR-Benchmark.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 37 of its 65 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

37 of 65 references.

References · 37

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2021Convex optimizationcited by 2,515
  4. 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,139
  5. 2017Neural Collaborative Filteringcited by 1,111
  6. 2015Image-based Recommendations on Styles and Substitutescited by 706
  7. 2020Self-supervised Learning: Generative or Contrastivecited by 694
  8. 2019Neural Graph Collaborative Filteringcited by 661
  9. 2020A Survey on Contrastive Self-supervised Learningcited by 554
  10. 2016Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filteringcited by 520
  11. 2020Graph Neural Networks in Recommender Systems: A Surveycited by 494
  12. 2020Self-supervised Graph Learning for Recommendationcited by 305
25 more
  1. 2020Bias and Debias in Recommender System: A Survey and Future Directionscited by 240
  2. 2017Neural Factorization Machines for Sparse Predictive Analyticscited by 183
  3. 2017Beyond triplet loss: a deep quadruplet network for person re-identificationcited by 172
  4. 2015VBPR: Visual Bayesian Personalized Ranking from Implicit Feedbackcited by 151
  5. 2020Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender Systemcited by 94
  6. 2021AutoDebias: Learning to Debias for Recommendationcited by 68
  7. 2022XSimGCL: Towards Extremely Simple Graph Contrastive Learning for Recommendationcited by 57
  8. 2021DORO: Distributional and Outlier Robust Optimizationcited by 56
  9. 2020On the Equivalence of Decoupled Graph Convolution Network and Label Propagationcited by 49
  10. 2024Macro Graph Neural Networks for Online Billion-Scale Recommender Systemscited by 40
  11. 2017Latent Relational Metric Learning via Memory-based Attention for Collaborative Rankingcited by 39
  12. 2022On the Effectiveness of Sampled Softmax Loss for Item Recommendationcited by 39
  13. 2020Graph Convolutional Network for Recommendation with Low-pass Collaborative Filterscited by 32
  14. 2021AutoLoss: Automated Loss Function Search in Recommendationscited by 31
  15. 2023Empowering Collaborative Filtering with Principled Adversarial Contrastive Losscited by 24
  16. 2021Popularity Bias Is Not Always Evil: Disentangling Benign and Harmful Bias for Recommendationcited by 23
  17. 2023Invariant Collaborative Filtering to Popularity Distribution Shiftcited by 23
  18. 2023Understanding Contrastive Learning via Distributionally Robust Optimizationcited by 21
  19. 2023Adap-τ: Adaptively Modulating Embedding Magnitude for Recommendationcited by 19
  20. 2024Distributionally Robust Graph-based Recommendation Systemcited by 17
  21. 2023Outlier-Robust Wasserstein DROcited by 16
  22. 2020Graph Convolution Machine for Context-aware Recommender Systemcited by 13
  23. 2024LLM4DSR: Leveraging Large Language Model for Denoising Sequential Recommendationcited by 10
  24. 2024Lower-Left Partial AUC: An Effective and Efficient Optimization Metric for Recommendationcited by 9
  25. 2023BSL: Understanding and Improving Softmax Loss for Recommendationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 7.

9 papers.

  1. 2025Rethinking Entropy Interventions in RLVR: An Entropy Change Perspectivecited by 52
  2. 2026Prototypicality Bias Reveals Blindspots in Multimodal Evaluation Metricscited by 2
  3. 2025Rejuvenating Cross-Entropy Loss in Knowledge Distillation for Recommender Systemscited by 1
  4. 2026Talos: Optimizing Top-K Accuracy in Recommender Systemscited by 1
  5. 2026BEAR: Towards Beam-Search-Aware Optimization for Recommendation with Large Language Modelscited by 1
  6. 2026Correct and Weight: A Simple Yet Effective Loss for Implicit Feedback Recommendationcited by 0
  7. 2026Is Softmax Loss All You Need? A Principled Analysis of Softmax-family Losscited by 0
  8. 2026DSL: Understanding and Improving Softmax Recommender Systems with Competition-Aware Scalingcited by 0
  9. 2026The Pitfall of Scaling Up: Uncovering and Mitigating Popularity Bias Amplification in Scaling Transformer-based Recommenderscited by 0