mlsanityData to 7 Oct 2026

Aligning Large Language Models for Controllable Recommendations

arXiv 2403.05063 ↗First posted 8 Mar 2024cs.IR cs.AIcited by 19

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG

Abstract (arXiv)

Inspired by the exceptional general intelligence of Large Language Models (LLMs), researchers have begun to explore their application in pioneering the next generation of recommender systems - systems that are conversational, explainable, and controllable. However, existing literature primarily concentrates on integrating domain-specific knowledge into LLMs to enhance accuracy, often neglecting the ability to follow instructions. To address this gap, we initially introduce a collection of supervised learning tasks, augmented with labels derived from a conventional recommender model, aimed at explicitly improving LLMs' proficiency in adhering to recommendation-specific instructions. Subsequently, we develop a reinforcement learning-based alignment procedure to further strengthen LLMs' aptitude in responding to users' intentions and mitigating formatting errors. Through extensive experiments on two real-world datasets, our method markedly advances the capability of LLMs to comply with instructions within recommender systems, while sustaining a high level of accuracy performance.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 14 of its 21 references that are in the map. First posted on arXiv 8 Mar 2024.

Builds on

14 of 21 references.

References · 14

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2018Self-Attentive Sequential Recommendationcited by 1,169
  6. 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 396
  7. 2023A Survey on Large Language Models for Recommendationcited by 378
  8. 2023TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendationcited by 336
  9. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
  10. 2023Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender Systemcited by 241
  11. 2023Is ChatGPT a Good Recommender? A Preliminary Studycited by 192
  12. 2023Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approachcited by 168
2 more
  1. 2023Uncovering ChatGPT's Capabilities in Recommender Systemscited by 160
  2. 2023PALR: Personalization Aware LLMs for Recommendationcited by 96

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 10; 2026: 4.

19 papers.

Sort:

By area:

  1. 2025Rec-R1: Bridging Generative Large Language Models and User-Centric Recommendation Systems via Reinforcement Learningcited by 32
  2. 2024Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyondcited by 26
  3. 2024RecAI: Leveraging Large Language Models for Next-Generation Recommender Systemscited by 20
  4. 2024A Survey of Generative Search and Recommendation in the Era of Large Language Modelscited by 14
  5. 2024A Survey of Controllable Learning: Methods and Applications in Information Retrievalcited by 12
  6. 2025Improving LLM-powered Recommendations with Personalized Informationcited by 9
  7. 2024Mitigating Propensity Bias of Large Language Models for Recommender Systemscited by 8
  8. 2025TrackRec: Iterative Alternating Feedback with Chain-of-Thought via Preference Alignment for Recommendationcited by 7
  9. 2025Eliminating Out-of-Domain Recommendations in LLM-based Recommender Systems: A Unified Viewcited by 5
  10. 2025Bidirectional Knowledge Distillation for Enhancing Sequential Recommendation with Large Language Modelscited by 4
  11. 2025LLM-Aligned Geographic Item Tokenization for Local-Life Recommendationcited by 3
  12. 2025Sparse Autoencoders for Sequential Recommendation Models: Interpretation and Flexible Controlcited by 2
7 more
  1. 2025Catalog-Native LLM: Speaking Item-ID Dialect with Less Entanglement for Recommendationcited by 1
  2. 2026Owen-Shapley Policy Optimization: A Principled RL Algorithm for Generative Search LLMscited by 1
  3. 2026Offline Reasoning for Efficient Recommendation: LLM-Empowered Persona-Profiled Item Indexingcited by 1
  4. 2025Optimizing Recommendations using Fine-Tuned LLMscited by 0
  5. 2025CLLMRec: LLM-powered Cognitive-Aware Concept Recommendation via Semantic Alignment and Prerequisite Knowledge Distillationcited by 0
  6. 2026Give Users the Wheel: Towards Promptable Recommendation Paradigmcited by 0
  7. 2026Dear Algo: A Precision-First Agentic Intent Layer for Unified Search and Recommendationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023RecExplainer: Aligning Large Language Models for Explaining Recommendation Models
  2. 2025RLHF Fine-Tuning of LLMs for Alignment with Implicit User Feedback in Conversational Recommenders
  3. 2024Behavior Alignment: A New Perspective of Evaluating LLM-based Conversational Recommender Systems
  4. 2024Semantic Convergence: Harmonizing Recommender Systems via Two-Stage Alignment and Behavioral Semantic Tokenization
  5. 2025Rank-GRPO: Training LLM-based Conversational Recommender Systems with Reinforcement Learning
  6. 2025A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challenges
  7. 2024Aligning Large Language Models with Recommendation Knowledge
  8. 2025User Feedback Alignment for LLM-powered Exploration in Large-scale Recommendation Systems