mlsanityData to 7 Oct 2026

Uncertainty Quantification and Decomposition for LLM-based Recommendation

arXiv 2501.17630 ↗First posted 29 Jan 2025cs.IR cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Despite the widespread adoption of large language models (LLMs) for recommendation, we demonstrate that LLMs often exhibit uncertainty in their recommendations. To ensure the trustworthy use of LLMs in generating recommendations, we emphasize the importance of assessing the reliability of recommendations generated by LLMs. We start by introducing a novel framework for estimating the predictive uncertainty to quantitatively measure the reliability of LLM-based recommendations. We further propose to decompose the predictive uncertainty into recommendation uncertainty and prompt uncertainty, enabling in-depth analyses of the primary source of uncertainty. Through extensive experiments, we (1) demonstrate predictive uncertainty effectively indicates the reliability of LLM-based recommendations, (2) investigate the origins of uncertainty with decomposed uncertainty measures, and (3) propose uncertainty-aware prompting for a lower predictive uncertainty and enhanced recommendation. Our source code and model weights are available at https://github.com/WonbinKweon/UNC_LLM_REC_WWW2025

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 45 of its 83 references that are in the map. First posted on arXiv 29 Jan 2025.

Builds on

45 of 83 references.

References · 45

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2017On Calibration of Modern Neural Networkscited by 4,860
  7. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  8. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  9. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  10. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  11. 2018Self-Attentive Sequential Recommendationcited by 1,169
  12. 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,134
33 more
  1. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 952
  2. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  3. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  4. 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 396
  5. 2023A Survey on Large Language Models for Recommendationcited by 378
  6. 2023TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendationcited by 336
  7. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
  8. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 302
  9. 2023Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender Systemcited by 241
  10. 2019Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacementcited by 205
  11. 2023Is ChatGPT a Good Recommender? A Preliminary Studycited by 192
  12. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  13. 2023Uncovering ChatGPT's Capabilities in Recommender Systemscited by 160
  14. 2023Representation Learning with Large Language Models for Recommendationcited by 155
  15. 2022M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systemscited by 152
  16. 2023Large Language Models as Zero-Shot Conversational Recommenderscited by 126
  17. 2022Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysiscited by 104
  18. 2023CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendationcited by 99
  19. 2023PALR: Personalization Aware LLMs for Recommendationcited by 96
  20. 2018Quantifying Uncertainties in Natural Language Processing Taskscited by 79
  21. 2023Leveraging Large Language Models for Sequential Recommendationcited by 78
  22. 2023Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based Preferencescited by 66
  23. 2023Collaborative Large Language Model for Recommender Systemscited by 63
  24. 2024Uncertainty Quantification for In-Context Learning of Large Language Modelscited by 45
  25. 2023Sensitivity and Robustness of Large Language Models to Prompt Template in Japanese Text Classification Taskscited by 25
  26. 2021Modeling Sequences as Distributions with Uncertainty for Sequential Recommendationcited by 22
  27. 2019Candidate Generation with Binary Codes for Large-Scale Top-N Recommendationcited by 19
  28. 2023Knowledge Plugins: Enhancing Large Language Models for Domain-Specific Recommendationscited by 16
  29. 2021Obtaining Calibrated Probabilities with Personalized Ranking Modelscited by 15
  30. 2020Convolutional Gaussian Embeddings for Personalized Recommendation with Uncertaintycited by 11
  31. 2022Uncertainty Quantification for Fairness in Two-Stage Recommender Systemscited by 10
  32. 2024Doubly Calibrated Estimator for Recommendation on Data Missing Not At Randomcited by 7
  33. 2024Aligning Language Models for Versatile Text-based Item Retrievalcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 7.

12 papers.

  1. 2025An overview of model uncertainty and variability in LLM-based sentiment analysis. Challenges, mitigation strategies and the role of explainabilitycited by 15
  2. 2025Learned Hallucination Detection in Black-Box LLMs using Token-level Entropy Production Ratecited by 9
  3. 2025Topic Coverage-based Demonstration Retrieval for In-Context Learningcited by 3
  4. 2025Continual Low-Rank Adapters for LLM-based Generative Recommender Systemscited by 2
  5. 2026Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Modelscited by 2
  6. 2026User-Adaptive Meta-Learning for Cold-Start Medication Recommendation with Uncertainty Filteringcited by 1
  7. 2026Uncertainty and Fairness Awareness in LLM-Based Recommendation Systemscited by 1
  8. 2026Do LLM Recommenders Know When They're Hallucinating? Auditing Confidence Calibration in Catalog Faithfulnesscited by 1
  9. 2025Large Language Models and 3D Vision for Intelligent Robotic Perception and Autonomycited by 0
  10. 2026A Risk Decomposition Framework for Pre-Hoc Fine-Tuning Predictioncited by 0
  11. 2026Ask to Be Sure: Informative Interactions for Confident Multi-Turn LLM Recommendationcited by 0
  12. 2026Profiling What Matters: Context-Aware Item Profiles from Large-Scale Metadata for LLM Recommenderscited by 0