mlsanityData to 14 Oct 2026

PrLM: Learning Explicit Reasoning for Personalized RAG via Contrastive Reward Optimization

arXiv 2508.07342 ↗First posted 10 Aug 2025cs.IR cs.CLcited by 8

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 2 more
Abstract (arXiv)

Personalized retrieval-augmented generation (RAG) aims to produce user-tailored responses by incorporating retrieved user profiles alongside the input query. Existing methods primarily focus on improving retrieval and rely on large language models (LLMs) to implicitly integrate the retrieved context with the query. However, such models are often sensitive to retrieval quality and may generate responses that are misaligned with user preferences. To address this limitation, we propose PrLM, a reinforcement learning framework that trains LLMs to explicitly reason over retrieved user profiles. Guided by a contrastively trained personalization reward model, PrLM effectively learns from user responses without requiring annotated reasoning paths. Experiments on three personalized text generation datasets show that PrLM outperforms existing methods and remains robust across varying numbers of retrieved profiles and different retrievers.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 24 of its 37 references that are in the map. First posted on arXiv 10 Aug 2025.

Builds on

24 of 37 references.

References · 24

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 578
  4. 2025LIMO: Less is More for Reasoningcited by 498
  5. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 384
  6. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  7. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  8. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  9. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
  10. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 262
  11. 2023Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrieverscited by 92
  12. 2023Teach LLMs to Personalize -- An Approach inspired by Writing Educationcited by 48
12 more
  1. 2023Integrating Summarization and Retrieval for Enhanced Personalization via Large Language Modelscited by 45
  2. 2025Rec-R1: Bridging Generative Large Language Models and User-Centric Recommendation Systems via Reinforcement Learningcited by 33
  3. 2025Retrieval Augmented Generation with Collaborative Filtering for Personalized Text Generationcited by 24
  4. 2024CitaLaw: Enhancing LLM with Citations in Legal Domaincited by 16
  5. 2025QE-RAG: A Robust Retrieval-Augmented Generation Benchmark for Query Entry Errorscited by 11
  6. 2025Legal Mathematical Reasoning with LLMs: Procedural Alignment through Two-Stage Reinforcement Learningcited by 7
  7. 2024Beyond Guilt: Legal Judgment Prediction with Trichotomous Reasoningcited by 6
  8. 2025Bridging Search and Recommendation through Latent Cross Reasoningcited by 5
  9. 2025Unified Generative Search and Recommendationcited by 4
  10. 2025An Explicit Syllogistic Legal Reasoning Framework for Large Language Modelscited by 2
  11. 2025Balancing Stylization and Truth via Disentangled Representation Steeringcited by 2
  12. 2025Benefit from Rich: Tackling Search Interaction Sparsity in Search Enhanced Recommendationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2025Tagging the Thought: Unlocking Personalization Reasoning via Reinforcement Learningcited by 3
  2. 2025Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMscited by 2
  3. 2026Training LLMs with Reinforcement Learning for Intent-Aware Personalized Question Answeringcited by 2
  4. 2026Preference-Aware Rubric Learning for Personalized Evaluationcited by 2
  5. 2026Improving User Privacy in Personalized Generation: Client-Side Retrieval-Augmented Modification of Server-Side Generated Speculationscited by 1
  6. 2026ReLay: Personalized LLM-Generated Plain-Language Summaries for Better Understanding, but at What Cost?cited by 0
  7. 2026Forgotten History or Test-of-Time? Retrospect and Prospect on RAG from an IR Perspectivecited by 0
  8. 2026Rubric-Aware On-Policy Self-Distillation for LLM Personalizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning to Reason for Multi-Step Retrieval of Personal Context in Personalized Question Answering
  2. 2024Optimization Methods for Personalizing Large Language Models through Retrieval Augmentation
  3. 2024Bridging the Preference Gap between Retrievers and LLMs
  4. 2024RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment
  5. 2026Optimizing User Profiles via Contextual Bandits for Retrieval-Augmented LLM Personalization
  6. 2024RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation
  7. 2025Reasoning-Enhanced Self-Training for Long-Form Personalized Text Generation
  8. 2025RAG-Zeval: Towards Robust and Interpretable Evaluation on RAG Responses through End-to-End Rule-Guided Reasoning