mlsanityData to 14 Oct 2026

From Prompting to Behavioral Alignment: Personalized LLM Judges for Recommendation Evaluation

arXiv 2608.11493 ↗First posted 11 Aug 2026cs.AI cs.LGcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG

Abstract (arXiv)

Traditional offline recommendation evaluation relies heavily on complex, manually maintained feature pipelines that are difficult to scale. While Large Language Models (LLMs) offer a promising alternative by predicting user engagement directly from raw text logs, empirical analysis in this study identifies a critical failure mode termed bidirectional rationalization. In a zero-shot setting, LLMs are found to convincingly argue for both positive and negative user engagement outcomes on the exact same item with identical evidence, highlighting the unreliability of off-the-shelf LLMs in predicting user engagement. To resolve this, we develop and apply a sequential behavioral alignment framework pairing fine-tuning with preference optimization over paired correct and counterfactual rationales. Evaluated on real-world homepage interaction logs, this aligned reasoning approach achieves a 32.19% lift in Macro-F1 score over the zero-shot baseline and matches the production feature-engineered baseline. The results demonstrate that behavioral alignment mitigates bidirectional rationalization while delivering human-interpretable reasoning traces without manual pipeline overhead.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 11 of its 17 references that are in the map. First posted on arXiv 11 Aug 2026.

Builds on

11 of 17 references.

References · 11

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  4. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  5. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  6. 2018Offline A/B testing for Recommender Systemscited by 96
  7. 2025OneRec-Think: In-Text Reasoning for Generative Recommendationcited by 51
  8. 2025Reason-to-Recommend: Using Interaction-of-Thought Reasoning to Enhance LLM Recommendationcited by 20
  9. 2025ThinkRec: Thinking-based recommendation via LLMcited by 11
  10. 2025When Two LLMs Debate, Both Think They'll Wincited by 10
  11. 2025Evaluating Podcast Recommendations with Profile-Aware LLM-as-a-Judgecited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Behavior Alignment: A New Perspective of Evaluating LLM-based Conversational Recommender Systems
  2. 2025Align3GR: Unified Multi-Level Alignment for LLM-based Generative Recommendation
  3. 2023RecExplainer: Aligning Large Language Models for Explaining Recommendation Models
  4. 2025Reason4Rec: Deliberative User Preference Alignment of Large Language Models for Recommendation
  5. 2025User Feedback Alignment for LLM-powered Exploration in Large-scale Recommendation Systems
  6. 2025Aligning Black-box Language Models with Human Judgments
  7. 2026AlignUSER: Human-Aligned LLM Agents via World Models for Recommender System Evaluation
  8. 2025What Makes LLMs Effective Sequential Recommenders? A Study on Preference Intensity and Temporal Context