# Are Longer Prompts Always Better? Prompt Selection in Large Language Models for Recommendation Systems

arXiv: https://arxiv.org/abs/2412.14454 · HTML: https://mlsanity.com/paper/2412.14454/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 38 references that are in the map.
- First posted on arXiv 19 Dec 2024.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications](https://mlsanity.com/paper/2402.07927/) (2024)
- [Is ChatGPT a Good Recommender? A Preliminary Study](https://mlsanity.com/paper/2304.10149/) (2023)
- [Large Language Models for Generative Recommendation: A Survey and Visionary Discussions](https://mlsanity.com/paper/2309.01157/) (2023)
- [Zero-Shot Next-Item Recommendation using Large Pretrained Language Models](https://mlsanity.com/paper/2304.03153/) (2023)
- [LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking](https://mlsanity.com/paper/2311.02089/) (2023)
- [A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks](https://mlsanity.com/paper/2407.12994/) (2024)
- [Evaluating ChatGPT as a Recommender System: A Rigorous Approach](https://mlsanity.com/paper/2309.03613/) (2023)
- [One Model for All: Large Language Models are Domain-Agnostic Recommendation Systems](https://mlsanity.com/paper/2310.14304/) (2023)
- [Tapping the Potential of Large Language Models as Recommender Systems: A Comprehensive Framework and Empirical Analysis](https://mlsanity.com/paper/2401.04997/) (2024)

## Built on by (most cited first, 6 of 6)

- [AutoData: A Multi-Agent System for Open Web Data Collection](https://mlsanity.com/paper/2505.15859/) (2025)
- [A Taxonomy of Prompt Defects in LLM Systems](https://mlsanity.com/paper/2509.14404/) (2025)
- [Why is prompting hard? Understanding prompts on binary sequence predictors](https://mlsanity.com/paper/2502.10760/) (2025)
- [Training-Free Active Learning Framework in Materials Science with Large Language Models](https://mlsanity.com/paper/2511.19730/) (2025)
- [CTG-Insight: A Multi-Agent Interpretable LLM Framework for Cardiotocography Analysis and Classification](https://mlsanity.com/paper/2507.22205/) (2025)
- [Performance of large language models in the optical diagnosis of colorectal polyps](https://mlsanity.com/paper/2608.07543/) (2026)

## Abstract (arXiv)

In large language models (LLM)-based recommendation systems (LLM-RSs), accurately predicting user preferences by leveraging the general knowledge of LLMs is possible without requiring extensive training data. By converting recommendation tasks into natural language inputs called prompts, LLM-RSs can efficiently solve issues that have been difficult to address due to data scarcity but are crucial in applications such as cold-start and cross-domain problems. However, when applying this in practice, selecting the prompt that matches tasks and data is essential. Although numerous prompts have been proposed in LLM-RSs and representing the target user in prompts significantly impacts recommendation accuracy, there are still no clear guidelines for selecting specific prompts. In this paper, we categorize and analyze prompts from previous research to establish practical prompt selection guidelines. Through 450 experiments with 90 prompts and five real-world datasets, we examined the relationship between prompts and dataset characteristics in recommendation accuracy. We found that no single prompt consistently outperforms others; thus, selecting prompts on the basis of dataset characteristics is crucial. Here, we propose a prompt selection method that achieves higher accuracy with minimal validation data. Because increasing the number of prompts to explore raises costs, we also introduce a cost-efficient strategy using high-performance and cost-efficient LLMs, significantly reducing exploration costs while maintaining high prediction accuracy. Our work offers valuable insights into the prompt selection, advancing accurate and efficient LLM-RSs.