# Generative Explore-Exploit: Training-free Optimization of Generative Recommender Systems using LLM Optimizers

arXiv: https://arxiv.org/abs/2406.05255 · HTML: https://mlsanity.com/paper/2406.05255/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 39 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Builds on (most cited first, 19 of 19)

- [Is ChatGPT a Good Recommender? A Preliminary Study](https://mlsanity.com/paper/2304.10149/) (2023)
- [Reinforcement learning based recommender systems: A survey](https://mlsanity.com/paper/2101.06286/) (2021)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [GPT4Rec: A Generative Framework for Personalized Recommendation and User Interests Interpretation](https://mlsanity.com/paper/2304.03879/) (2023)
- [CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendation](https://mlsanity.com/paper/2310.19488/) (2023)
- [WebCPM: Interactive Web Search for Chinese Long-form Question Answering](https://mlsanity.com/paper/2305.06849/) (2023)
- [Generative Recommendation: Towards Next-generation Recommender Paradigm](https://mlsanity.com/paper/2304.03516/) (2023)
- [Improving Conversational Recommendation Systems' Quality with Context-Aware Item Meta Information](https://mlsanity.com/paper/2112.08140/) (2021)
- [MuseChat: A Conversational Music Recommendation System for Videos](https://mlsanity.com/paper/2310.06282/) (2023)
- [Question Generation for Reading Comprehension Assessment by Modeling How and What to Ask](https://mlsanity.com/paper/2204.02908/) (2022)
- [One Model for All: Large Language Models are Domain-Agnostic Recommendation Systems](https://mlsanity.com/paper/2310.14304/) (2023)
- [CTRL: Connect Collaborative and Language Model for CTR Prediction](https://mlsanity.com/paper/2306.02841/) (2023)
- [CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training](https://mlsanity.com/paper/2110.07731/) (2021)
- [Prompt Tuning Large Language Models on Personalized Aspect Extraction for Recommendations](https://mlsanity.com/paper/2306.01475/) (2023)
- [Generating Usage-related Questions for Preference Elicitation in Conversational Recommender Systems](https://mlsanity.com/paper/2111.13463/) (2021)
- [TF-DCon: Leveraging Large Language Models (LLMs) to Empower Training-Free Dataset Condensation for Content-Based Recommendation](https://mlsanity.com/paper/2310.09874/) (2023)
- [InstructPTS: Instruction-Tuning LLMs for Product Title Summarization](https://mlsanity.com/paper/2310.16361/) (2023)
- [Question Suggestion for Conversational Shopping Assistants Using Product Metadata](https://mlsanity.com/paper/2405.01738/) (2024)
- [Follow-on Question Suggestion via Voice Hints for Voice Assistants](https://mlsanity.com/paper/2310.17034/) (2023)

## Built on by (most cited first, 7 of 7)

- [Bridging the Gap Between Information Seeking and Product Search Systems: Q&A Recommendation for E-commerce](https://mlsanity.com/paper/2407.09653/) (2024)
- [Identifying High Consideration E-Commerce Search Queries](https://mlsanity.com/paper/2410.13951/) (2024)
- [JRE-L: Journalist, Reader, and Editor LLMs in the Loop for Science Journalism for the General Audience](https://mlsanity.com/paper/2501.16865/) (2025)
- [REIC: RAG-Enhanced Intent Classification at Scale](https://mlsanity.com/paper/2506.00210/) (2025)
- [Hint-Augmented Re-ranking: Efficient Product Search using LLM-Based Query Decomposition](https://mlsanity.com/paper/2511.13994/) (2025)
- [On the Memorization Behavior of LLMs in Generative Recommendation: Observations, Implications, and Training Strategies](https://mlsanity.com/paper/2606.17276/) (2026)
- [How Can Recommendation Feedback Evolve Agent Memory?](https://mlsanity.com/paper/2609.37544/) (2026)

## Abstract (arXiv)

Recommender systems are widely used to suggest engaging content, and Large Language Models (LLMs) have given rise to generative recommenders. Such systems can directly generate items, including for open-set tasks like question suggestion. While the world knowledge of LLMs enable good recommendations, improving the generated content through user feedback is challenging as continuously fine-tuning LLMs is prohibitively expensive. We present a training-free approach for optimizing generative recommenders by connecting user feedback loops to LLM-based optimizers. We propose a generative explore-exploit method that can not only exploit generated items with known high engagement, but also actively explore and discover hidden population preferences to improve recommendation quality. We evaluate our approach on question generation in two domains (e-commerce and general knowledge), and model user feedback with Click Through Rate (CTR). Experiments show our LLM-based explore-exploit approach can iteratively improve recommendations, and consistently increase CTR. Ablation analysis shows that generative exploration is key to learning user preferences, avoiding the pitfalls of greedy exploit-only approaches. A human evaluation strongly supports our quantitative findings.