# Enhancing Rating Prediction with Off-the-Shelf LLMs Using In-Context User Reviews

arXiv: https://arxiv.org/abs/2510.00449 · HTML: https://mlsanity.com/paper/2510.00449/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 32 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Builds on (most cited first, 12 of 12)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning](https://mlsanity.com/paper/2409.12183/) (2024)
- [Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction](https://mlsanity.com/paper/2305.06474/) (2023)
- [Personalization of Large Language Models: A Survey](https://mlsanity.com/paper/2411.00027/) (2024)
- [Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based Preferences](https://mlsanity.com/paper/2307.14225/) (2023)
- [Generating Personalized Recipes from Historical User Preferences](https://mlsanity.com/paper/1909.00105/) (2019)
- [Learning Personalized Alignment for Evaluating Open-ended Text Generation](https://mlsanity.com/paper/2310.03304/) (2023)
- [Large Language Models as Narrative-Driven Recommenders](https://mlsanity.com/paper/2410.13604/) (2024)

## Built on by (most cited first, 1 of 1)

- [What sentiment analysis can't see: Measuring whether customers were helped, and what went wrong, across 70,000 support conversations](https://mlsanity.com/paper/2606.19698/) (2026)

## Abstract (arXiv)

Personalizing the outputs of large language models (LLMs) to align with individual user preferences is an active research area. However, previous studies have mainly focused on classification or ranking tasks and have not considered Likert-scale rating prediction, a regression task that requires both language and mathematical reasoning to be solved effectively. This task has significant industrial applications, but the utilization of LLMs remains underexplored, particularly regarding the capabilities of off-the-shelf LLMs. This study investigates the performance of off-the-shelf LLMs on rating prediction, providing different in-context information. Through comprehensive experiments with eight models across three datasets, we demonstrate that user-written reviews significantly improve the rating prediction performance of LLMs. This result is comparable to traditional methods like matrix factorization, highlighting the potential of LLMs as a promising solution for the cold-start problem. We also find that the reviews for concrete items are more effective than general preference descriptions that are not based on any specific item. Furthermore, we discover that prompting LLMs to first generate a hypothetical review enhances the rating prediction performance. Our code is available at https://github.com/ynklab/rating-prediction-with-reviews.