mlsanityData to 7 Oct 2026

Factual and Personalized Recommendations using Language Models and Reinforcement Learning

arXiv 2310.06176 ↗First posted 9 Oct 2023cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Recommender systems (RSs) play a central role in connecting users to content, products, and services, matching candidate items to users based on their preferences. While traditional RSs rely on implicit user feedback signals, conversational RSs interact with users in natural language. In this work, we develop a comPelling, Precise, Personalized, Preference-relevant language model (P4LM) that recommends items to users while putting emphasis on explaining item characteristics and their relevance. P4LM uses the embedding space representation of a user's preferences to generate compelling responses that are factually-grounded and relevant w.r.t. the user's preferences. Moreover, we develop a joint reward function that measures precision, appeal, and personalization, which we use as AI-based feedback in a reinforcement learning-based language model framework. Using the MovieLens 25M dataset, we demonstrate that P4LM delivers compelling, personalized movie narratives to users.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 45 of its 75 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

45 of 75 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  6. 2017Deep reinforcement learning from human preferencescited by 3,846
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  8. 2020Learning to summarize from human feedbackcited by 2,562
  9. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  10. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  11. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  12. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
33 more
  1. 2017Neural Collaborative Filteringcited by 1,108
  2. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  3. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  4. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  5. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  6. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  7. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  8. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  9. 2016Deep Reinforcement Learning for Dialogue Generationcited by 286
  10. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  11. 2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 244
  12. 2020Bias and Debias in Recommender System: A Survey and Future Directionscited by 239
  13. 2018Explainable Recommendation: A Survey and New Perspectivescited by 210
  14. 2018Towards Deep Conversational Recommendationscited by 174
  15. 2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
  16. 2019Fairness in Recommendation Ranking through Pairwise Comparisonscited by 127
  17. 2020Neural Collaborative Filtering vs. Matrix Factorization Revisitedcited by 123
  18. 2018Conversational Recommender Systemcited by 115
  19. 2019Towards Knowledge-Based Recommender Dialog Systemcited by 112
  20. 2020Towards Topic-Guided Conversational Recommender Systemcited by 94
  21. 2019Distilling Policy Distillationcited by 81
  22. 2023Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedbackcited by 71
  23. 2023Leveraging Large Language Models in Conversational Recommender Systemscited by 63
  24. 2021Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarizationcited by 50
  25. 2019PRINCE: Provider-side Interpretability with Counterfactual Explanations in Recommender Systemscited by 35
  26. 2023TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterestcited by 35
  27. 2023The In-Sample Softmax for Offline Reinforcement Learningcited by 33
  28. 2023On the Independence of Association Bias and Empirical Fairness in Language Modelscited by 29
  29. 2018Personalized Language Model for Query Auto-Completioncited by 25
  30. 2021MORAL: Aligning AI with Human Norms through Multi-Objective Reinforced Active Learningcited by 24
  31. 2022The Expertise Problem: Learning from Specialized Feedbackcited by 20
  32. 2019The Impact of Popularity Bias on Fairness and Calibration in Recommendationcited by 9
  33. 2016Sample-efficient Deep Reinforcement Learning for Dialog Controlcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2025Rec-R1: Bridging Generative Large Language Models and User-Centric Recommendation Systems via Reinforcement Learningcited by 32
  2. 2024Embedding-Aligned Language Modelscited by 9
  3. 2025RecLLM-R1: A Two-Stage Training Paradigm with Reinforcement Learning and Chain-of-Thought v1cited by 6
  4. 2026Owen-Shapley Policy Optimization: A Principled RL Algorithm for Generative Search LLMscited by 1