Exploring Diversity, Novelty, and Popularity Bias in ChatGPT's Recommendations
Not on the map. Why?
Abstract (arXiv)
ChatGPT has emerged as a versatile tool, demonstrating capabilities across diverse domains. Given these successes, the Recommender Systems (RSs) community has begun investigating its applications within recommendation scenarios primarily focusing on accuracy. While the integration of ChatGPT into RSs has garnered significant attention, a comprehensive analysis of its performance across various dimensions remains largely unexplored. Specifically, the capabilities of providing diverse and novel recommendations or exploring potential biases such as popularity bias have not been thoroughly examined. As the use of these models continues to expand, understanding these aspects is crucial for enhancing user satisfaction and achieving long-term personalization. This study investigates the recommendations provided by ChatGPT-3.5 and ChatGPT-4 by assessing ChatGPT's capabilities in terms of diversity, novelty, and popularity bias. We evaluate these models on three distinct datasets and assess their performance in Top-N recommendation and cold-start scenarios. The findings reveal that ChatGPT-4 matches or surpasses traditional recommenders, demonstrating the ability to balance novelty and diversity in recommendations. Furthermore, in the cold-start scenario, ChatGPT models exhibit superior performance in both accuracy and novelty, suggesting they can be particularly beneficial for new users. This research highlights the strengths and limitations of ChatGPT's recommendations, offering new perspectives on the capacity of these models to provide recommendations beyond accuracy-focused metrics.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 79 references that are in the map. First posted on arXiv 5 Jan 2026.
Builds on
42 of 79 references.
References · 42
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
- 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,134
- 2017Neural Collaborative Filteringcited by 1,108
- 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
- 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
- 2023Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender Systemcited by 241
30 more
- 2023Better Zero-Shot Reasoning with Role-Play Promptingcited by 220
- 2023Is ChatGPT a Good Recommender? A Preliminary Studycited by 192
- 2023GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networkscited by 166
- 2023Uncovering ChatGPT's Capabilities in Recommender Systemscited by 160
- 2020Neural Collaborative Filtering vs. Matrix Factorization Revisitedcited by 123
- 2023Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendationcited by 113
- 2019Embarrassingly Shallow Autoencoders for Sparse Datacited by 93
- 2019Managing Popularity Bias in Recommender Systems with Personalized Re-rankingcited by 81
- 2023Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based Preferencescited by 66
- 2013Empirical Analysis of Predictive Algorithms for Collaborative Filteringcited by 47
- 2024Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Modelscited by 44
- 2023gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Samplingcited by 29
- 2021Elliot: a Comprehensive and Rigorous Framework for Reproducible Recommender Systems Evaluationcited by 28
- 2024Understanding Biases in ChatGPT-based Recommender Systems: Provider Fairness, Temporal Stability, and Recencycited by 27
- 2023BookGPT: A General Framework for Book Recommendation Empowered by Large Language Modelcited by 22
- 2024Enhancing Recommendation Diversity by Re-ranking with Large Language Modelscited by 19
- 2025LLaMAs Have Feelings Too: Unveiling Sentiment and Emotion Representations in LLaMA Models Through Probingcited by 14
- 2025Do LLMs Memorize Recommendation Datasets? A Preliminary Study on MovieLens-1Mcited by 12
- 2025Are the Hidden States Hiding Something? Testing the Limits of Factuality-Encoding Capabilities in LLMscited by 11
- 2024Popularity-Aware Alignment and Contrast for Mitigating Popularity Biascited by 9
- 2023Provider Fairness and Beyond-Accuracy Trade-offs in Recommender Systemscited by 6
- 2023Beyond-Accuracy: A Review on Diversity, Serendipity and Fairness in Recommender Systems Based on Graph Neural Networkscited by 6
- 2023Lending Interaction Wings to Recommender Systems with Conversational Agentscited by 6
- 2024Ducho 2.0: Towards a More Up-to-Date Unified Framework for the Extraction of Multimodal Features in Recommendationcited by 4
- 2024Can Large Language Models Assess Serendipity in Recommender Systems?cited by 3
- 2024DataRec: A Python Library for Standardized and Reproducible Data Management in Recommender Systemscited by 2
- 2025Training-Free, Identity-Preserving Image Editing for Fashion Pose Alignment and Normalizationcited by 2
- 2025Do We Really Need Specialization? Evaluating Generalist Text Embeddings for Zero-Shot Recommendation and Searchcited by 2
- 2023Post-hoc Selection of Pareto-Optimal Solutions in Search and Recommendationcited by 1
- 2024KGUF: Simple Knowledge-aware Graph-based Recommender with User-based Semantic Features Filteringcited by 1
Built on by
Not cited yet.