mlsanityData to 7 Oct 2026

PREDILECT: Preferences Delineated with Zero-Shot Language-based Reasoning in Reinforcement Learning

arXiv 2402.15420 ↗First posted 23 Feb 2024cs.RO cs.CL cs.LGcited by 8

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Preference-based reinforcement learning (RL) has emerged as a new field in robot learning, where humans play a pivotal role in shaping robot behavior by expressing preferences on different sequences of state-action pairs. However, formulating realistic policies for robots demands responses from humans to an extensive array of queries. In this work, we approach the sample-efficiency challenge by expanding the information collected per query to contain both preferences and optional text prompting. To accomplish this, we leverage the zero-shot capabilities of a large language model (LLM) to reason from the text provided by humans. To accommodate the additional query information, we reformulate the reward learning objectives to contain flexible highlights -- state-action pairs that contain relatively high information and are related to the features processed in a zero-shot fashion from a pretrained LLM. In both a simulated scenario and a user study, we reveal the effectiveness of our work by analyzing the feedback and its implications. Additionally, the collective feedback collected serves to train a robot on socially compliant trajectories in a simulated social navigation landscape. We provide video examples of the trained policies at https://sites.google.com/view/rl-predilect

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 18 of its 80 references that are in the map. First posted on arXiv 23 Feb 2024.

Builds on

18 of 80 references.

Closest ancestors · 1

Same direction, newest first

  1. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  4. 2016OpenAI Gymcited by 1,684
  5. 2016Concrete Problems in AI Safetycited by 1,582
  6. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  7. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  8. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  9. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  10. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
  11. 2022Large Language Models Are Reasoning Teacherscited by 318
  12. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
5 more
  1. 2016Learning to Navigate in Complex Environmentscited by 192
  2. 2016Loss is its own Reward: Self-Supervision for Reinforcement Learningcited by 72
  3. 2022Causal Confusion and Reward Misidentification in Preference-Based Reward Learningcited by 52
  4. 2023Principles and Guidelines for Evaluating Social Robot Navigation Algorithmscited by 52
  5. 2021Natural Language for Human-Robot Collaboration: Problems Beyond Language Groundingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 4; 2026: 2.

8 papers.

  1. 2025The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learningcited by 10
  2. 2024ICPL: Few-shot In-context Preference Learning via LLMscited by 7
  3. 2024Leveraging Sub-Optimal Data for Human-in-the-Loop Reinforcement Learningcited by 6
  4. 2025Long-Term Planning Around Humans in Domestic Environments with 3D Scene Graphscited by 4
  5. 2025FLoRA: Sample-Efficient Preference-based RL via Low-Rank Style Adaptation of Reward Functionscited by 3
  6. 2026Causally Robust Reward Learning from Reason-Augmented Preference Feedbackcited by 2
  7. 2025The Impact of VR and 2D Interfaces on Human Feedback in Preference-Based Robot Learningcited by 1
  8. 2026Robust Instruction Compliance in Cooperative Multi-Agent Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025QuickLAP: Quick Language-Action Preference Learning for Semi-Autonomous Agents
  2. 2024PrefCLM: Enhancing Preference-based Reinforcement Learning with Crowdsourced Large Language Models ↗
  3. 2025Efficiently Generating Expressive Quadruped Behaviors via Language-Guided Preference Learning
  4. 2026A Regret Minimization Framework on Preference Learning in Large Language Models
  5. 2026From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models
  6. 2025VARP: Reinforcement Learning from Vision-Language Model Feedback with Agent Regularized Preferences
  7. 2025LAPP: Large Language Model Feedback for Preference-Driven Reinforcement Learning
  8. 2026Interpreting Context-Aware Human Preferences for Multi-Objective Robot Navigation ↗