mlsanityData to 7 Oct 2026

Is ChatGPT a Good Recommender? A Preliminary Study

arXiv 2304.10149 ↗First posted 20 Apr 2023cs.IRcited by 192

Not on the map. Why?

Abstract (arXiv)

Recommendation systems have witnessed significant advancements and have been widely used over the past decades. However, most traditional recommendation methods are task-specific and therefore lack efficient generalization ability. Recently, the emergence of ChatGPT has significantly advanced NLP tasks by enhancing the capabilities of conversational models. Nonetheless, the application of ChatGPT in the recommendation domain has not been thoroughly investigated. In this paper, we employ ChatGPT as a general-purpose recommendation model to explore its potential for transferring extensive linguistic and world knowledge acquired from large-scale corpora to recommendation scenarios. Specifically, we design a set of prompts and evaluate ChatGPT's performance on five recommendation scenarios. Unlike traditional recommendation methods, we do not fine-tune ChatGPT during the entire evaluation process, relying only on the prompts themselves to convert recommendation tasks into natural language tasks. Further, we explore the use of few-shot prompting to inject interaction information that contains user potential interest to help ChatGPT better understand user needs and interests. Comprehensive experimental results on Amazon Beauty dataset show that ChatGPT has achieved promising results in certain tasks and is capable of reaching the baseline level in others. We conduct human evaluations on two explainability-oriented tasks to more accurately evaluate the quality of contents generated by different models. And the human evaluations show ChatGPT can truly understand the provided information and generate clearer and more reasonable results. We hope that our study can inspire researchers to further explore the potential of language models like ChatGPT to improve recommendation performance and contribute to the advancement of the recommendation systems field.

Key facts. Cited by 192 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Apr 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 47; 2024: 76; 2025: 44; 2026: 25.

192 papers.

Sort:

By area:

  1. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  2. 2023A Survey on Large Language Models for Recommendationcited by 378
  3. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
  4. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  5. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  6. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 203
  7. 2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 188
  8. 2023Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approachcited by 168
  9. 2023Representation Learning with Large Language Models for Recommendationcited by 155
  10. 2023How Can Recommender Systems Benefit from Large Language Models: A Surveycited by 141
  11. 2023Recommender AI Agent: Integrating Large Language Models for Interactive Recommendationscited by 130
  12. 2023RecMind: Large Language Model Powered Agent For Recommendationcited by 129
38 more
  1. 2023Large Language Models as Zero-Shot Conversational Recommenderscited by 126
  2. 2023AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systemscited by 116
  3. 2023Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Predictioncited by 113
  4. 2023PALR: Personalization Aware LLMs for Recommendationcited by 96
  5. 2023Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Datasetcited by 90
  6. 2023When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunitiescited by 89
  7. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  8. 2023LLM-Rec: Personalized Recommendation via Prompting Large Language Modelscited by 77
  9. 2023ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendationcited by 76
  10. 2024User-LLM: Efficient LLM Contextualization with User Embeddingscited by 74
  11. 2023Large Language Models for Generative Recommendation: A Survey and Visionary Discussionscited by 72
  12. 2023Penetrative AI: Making LLMs Comprehend the Physical Worldcited by 64
  13. 2024On Softmax Direct Preference Optimization for Recommendationcited by 64
  14. 2024HYDRA: Model Factorization Framework for Black-Box LLM Personalizationcited by 59
  15. 2024A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)cited by 58
  16. 2025RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMscited by 58
  17. 2023ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Modelscited by 54
  18. 2023LlamaRec: Two-Stage Recommendation using Large Language Models for Rankingcited by 52
  19. 2023Fine-tuning ChatGPT for Automatic Scoringcited by 50
  20. 2024IDGenRec: LLM-RecSys Alignment with Textual ID Learningcited by 50
  21. 2023Can Large Language Models Empower Molecular Property Prediction?cited by 43
  22. 2023Knowledge-Augmented Large Language Models for Personalized Contextual Query Suggestioncited by 43
  23. 2023Amazon-M2: A Multilingual Multi-locale Shopping Session Dataset for Recommendation and Text Generationcited by 42
  24. 2025A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applicationscited by 40
  25. 2023E4SRec: An Elegant Effective Efficient Extensible Solution of Large Language Models for Sequential Recommendationcited by 39
  26. 2024TravelAgent: An AI Assistant for Personalized Travel Planningcited by 38
  27. 2023LogGPT: Exploring ChatGPT for Log-Based Anomaly Detectioncited by 33
  28. 2024Do LLM Agents Have Regret? A Case Study in Online Learning and Gamescited by 33
  29. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  30. 2025PersonaX: A Recommendation Agent Oriented User Modeling Framework for Long Behavior Sequencecited by 33
  31. 2024Customizing Language Models with Instance-wise LoRA for Sequential Recommendationcited by 32
  32. 2023RAH! RecSys-Assistant-Human: A Human-Centered Recommendation Framework with LLM Agentscited by 30
  33. 2023LLMRec: Benchmarking Large Language Models on Recommendation Taskcited by 29
  34. 2023ClickPrompt: CTR Models are Strong Prompt Generators for Adapting Language Models to CTR Predictioncited by 27
  35. 2023DRDT: Dynamic Reflection with Divergent Thinking for LLM-based Sequential Recommendationcited by 27
  36. 2024Rethinking Large Language Model Architectures for Sequential Recommendationscited by 27
  37. 2025A Survey of Personalization: From RAG to Agentcited by 26
  38. 2024Can Small Language Models be Good Reasoners for Sequential Recommendation?cited by 25

50 most cited of 192 All 192 (JSON)