PREDICT: Preference Reasoning by Evaluating Decomposed preferences Inferred from Candidate Trajectories
On the map: LLM reasoning → Reinforcement learning for reasoning
Abstract (arXiv)
Accommodating human preferences is essential for creating AI agents that deliver personalized and effective interactions. Recent work has shown the potential for LLMs to infer preferences from user interactions, but they often produce broad and generic preferences, failing to capture the unique and individualized nature of human preferences. This paper introduces PREDICT, a method designed to enhance the precision and adaptability of inferring preferences. PREDICT incorporates three key elements: (1) iterative refinement of inferred preferences, (2) decomposition of preferences into constituent components, and (3) validation of preferences across multiple trajectories. We evaluate PREDICT on two distinct environments: a gridworld setting and a new text-domain environment (PLUME). PREDICT more accurately infers nuanced human preferences improving over existing baselines by 66.2% (gridworld environment) and 41.0% (PLUME).
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 7 of its 34 references that are in the map. First posted on arXiv 8 Oct 2024.
Builds on
7 of 34 references.
References · 7
- 2023LaMP: When Large Language Models Meet Personalizationcited by 437
- 2020Language Conditioned Imitation Learning over Unstructured Datacited by 209
- 2024Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuningcited by 90
- 2024Aligning LLM Agents by Learning Latent Preference from User Editscited by 69
- 2024HYDRA: Model Factorization Framework for Black-Box LLM Personalizationcited by 60
- 2024Prompt Optimization with Human Feedbackcited by 35
- 2024PMG : Personalized Multimodal Generation with Large Language Modelscited by 19
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.
- 2025LAPP: Large Language Model Feedback for Preference-Driven Reinforcement Learningcited by 2
- 2026Principled Fine-tuning of LLMs from User-Edits: A Medley of Preference, Supervision, and Rewardcited by 0
- 2026SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RLcited by 0
- 2026SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RL1 Oct 2026
- 2026Principled Fine-tuning of LLMs from User-Edits: A Medley of Preference, Supervision, and Reward27 Jan 2026
- 2025LAPP: Large Language Model Feedback for Preference-Driven Reinforcement Learning21 Apr 2025
Similar, not cited
Close in wording, no citation either way
- 2025Learning a Canonical Basis of Human Preferences from Binary Ratings
- 2025Modeling and Optimizing User Preferences in AI Copilots: A Comprehensive Survey and Taxonomy
- 2024Hybrid Preferences: Learning to Route Instances for Human vs. AI Feedback
- 2025Extended Inductive Reasoning for Personalized Preference Inference from Behavioral Signals
- 2026Learning Transferable Latent User Preferences for Human-Aligned Decision Making
- 2023Active Preference Inference using Language Models and Probabilistic Reasoning
- 2024Data-Centric Human Preference with Rationales for Direct Preference Alignment
- 2025WorldPM: Scaling Human Preference Modeling