# User Feedback Alignment for LLM-powered Exploration in Large-scale Recommendation Systems

arXiv: https://arxiv.org/abs/2504.05522 · HTML: https://mlsanity.com/paper/2504.05522/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2025.

## Built on by (most cited first, 6 of 6)

- [Evaluating Podcast Recommendations with Profile-Aware LLM-as-a-Judge](https://mlsanity.com/paper/2508.08777/) (2025)
- [A Metric for MLLM Alignment in Large-scale Recommendation](https://mlsanity.com/paper/2508.04963/) (2025)
- [IceBreaker for Conversational Agents: Breaking the First-Message Barrier with Personalized Starters](https://mlsanity.com/paper/2604.18375/) (2026)
- [Toward User Preference Alignment in LLM Recommendation via Explicit Context Feedback](https://mlsanity.com/paper/2605.29141/) (2026)
- [When Relevance Meets Novelty: Dual-Stable Periodic Optimization for Serendipitous Recommendation](https://mlsanity.com/paper/2508.00450/) (2025)
- [PersonaMem-v3: Toward Omni-Platform Personal Intelligence for Holistic User Understanding, Recommendation, and Agentic Tasks](https://mlsanity.com/paper/2608.21381/) (2026)

## Abstract (arXiv)

Exploration, the act of broadening user experiences beyond their established preferences, is challenging in large-scale recommendation systems due to feedback loops and limited signals on user exploration patterns. Large Language Models (LLMs) offer potential solutions by leveraging their world knowledge to recommend novel content outside these loops. A key challenge is aligning LLMs with user preferences while preserving their knowledge and reasoning. To enhance planning for new user interests using LLMs, this paper introduces a novel approach that combines hierarchical planning with LLM inference-time scaling. This method aims to improve recommendation relevancy without compromising novelty. We decouple novelty and user-alignment, training separate LLMs for each objective. We then scale up the novelty-focused LLM's inference and select the best-of-n predictions using the user-aligned LLM. Live experiments demonstrate efficacy, showing significant gains in both user satisfaction (measured by watch activity and active user counts) and exploration diversity.