# Tapping the Potential of Large Language Models as Recommender Systems: A Comprehensive Framework and Empirical Analysis

arXiv: https://arxiv.org/abs/2401.04997 · HTML: https://mlsanity.com/paper/2401.04997/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jan 2024.

## Built on by (most cited first, 17 of 17)

- [Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era](https://mlsanity.com/paper/2404.11457/) (2024)
- [Large Language Models as Recommender Systems: A Study of Popularity Bias](https://mlsanity.com/paper/2406.01285/) (2024)
- [Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyond](https://mlsanity.com/paper/2410.19744/) (2024)
- [Towards Efficient and Effective Unlearning of Large Language Models for Recommendation](https://mlsanity.com/paper/2403.03536/) (2024)
- [R4ec: A Reasoning, Reflection, and Refinement Framework for Recommendation Systems](https://mlsanity.com/paper/2507.17249/) (2025)
- [Exploring the Impact of Large Language Models on Recommender Systems: An Extensive Review](https://mlsanity.com/paper/2402.18590/) (2024)
- [Are Longer Prompts Always Better? Prompt Selection in Large Language Models for Recommendation Systems](https://mlsanity.com/paper/2412.14454/) (2024)
- [Privacy Risks of LLM-Empowered Recommender Systems: An Inversion Attack Perspective](https://mlsanity.com/paper/2508.03703/) (2025)
- [Preserving Privacy and Utility in LLM-Based Product Recommendations](https://mlsanity.com/paper/2505.00951/) (2025)
- [Large Language Models as Conversational Movie Recommenders: A User Study](https://mlsanity.com/paper/2404.19093/) (2024)
- [Eval4DiRec: A Unified and Systematic Evaluation Framework for Diffusion-based Recommender Systems](https://mlsanity.com/paper/2609.34404/) (2026)
- [See Where You Read with Eye Gaze Tracking and Large Language Model](https://mlsanity.com/paper/2409.19454/) (2024)
- [GRAB: An LLM-Inspired Sequence-First Click-Through Rate Prediction Modeling Paradigm](https://mlsanity.com/paper/2602.01865/) (2026)
- [Give Users the Wheel: Towards Promptable Recommendation Paradigm](https://mlsanity.com/paper/2602.18929/) (2026)
- [Fair Agents: Balancing Multistakeholder Alignment in Multi-Agent Personalization Systems](https://mlsanity.com/paper/2605.02379/) (2026)
- [Dialogue to Discovery: Attribute-Aware Preference Elicitation for Conversational Product Search Assistants](https://mlsanity.com/paper/2606.24194/) (2026)
- [Tokenizing Numerical and Embedding Features for LLM RecSys](https://mlsanity.com/paper/2607.10016/) (2026)

## Abstract (arXiv)

Recently, Large Language Models~(LLMs) such as ChatGPT have showcased remarkable abilities in solving general tasks, demonstrating the potential for applications in recommender systems. To assess how effectively LLMs can be used in recommendation tasks, our study primarily focuses on employing LLMs as recommender systems through prompting engineering. We propose a general framework for utilizing LLMs in recommendation tasks, focusing on the capabilities of LLMs as recommenders. To conduct our analysis, we formalize the input of LLMs for recommendation into natural language prompts with two key aspects, and explain how our framework can be generalized to various recommendation scenarios. As for the use of LLMs as recommenders, we analyze the impact of public availability, tuning strategies, model architecture, parameter scale, and context length on recommendation results based on the classification of LLMs. As for prompt engineering, we further analyze the impact of four important components of prompts, ie task descriptions, user interest modeling, candidate items construction and prompting strategies. In each section, we first define and categorize concepts in line with the existing literature. Then, we propose inspiring research questions followed by detailed experiments on two public datasets, in order to systematically analyze the impact of different factors on performance. Based on our empirical analysis, we finally summarize promising directions to shed lights on future research.