# PREDICT: Preference Reasoning by Evaluating Decomposed preferences Inferred from Candidate Trajectories

arXiv: https://arxiv.org/abs/2410.06273 · HTML: https://mlsanity.com/paper/2410.06273/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 7 of its 34 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 7 of 7)

- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [Language Conditioned Imitation Learning over Unstructured Data](https://mlsanity.com/paper/2005.07648/) (2020)
- [Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning](https://mlsanity.com/paper/2402.04401/) (2024)
- [Aligning LLM Agents by Learning Latent Preference from User Edits](https://mlsanity.com/paper/2404.15269/) (2024)
- [HYDRA: Model Factorization Framework for Black-Box LLM Personalization](https://mlsanity.com/paper/2406.02888/) (2024)
- [Prompt Optimization with Human Feedback](https://mlsanity.com/paper/2405.17346/) (2024)
- [PMG : Personalized Multimodal Generation with Large Language Models](https://mlsanity.com/paper/2404.08677/) (2024)

## Built on by (most cited first, 3 of 3)

- [LAPP: Large Language Model Feedback for Preference-Driven Reinforcement Learning](https://mlsanity.com/paper/2504.15472/) (2025)
- [Principled Fine-tuning of LLMs from User-Edits: A Medley of Preference, Supervision, and Reward](https://mlsanity.com/paper/2601.19055/) (2026)
- [SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RL](https://mlsanity.com/paper/2610.02023/) (2026)

## Abstract (arXiv)

Accommodating human preferences is essential for creating AI agents that deliver personalized and effective interactions. Recent work has shown the potential for LLMs to infer preferences from user interactions, but they often produce broad and generic preferences, failing to capture the unique and individualized nature of human preferences. This paper introduces PREDICT, a method designed to enhance the precision and adaptability of inferring preferences. PREDICT incorporates three key elements: (1) iterative refinement of inferred preferences, (2) decomposition of preferences into constituent components, and (3) validation of preferences across multiple trajectories. We evaluate PREDICT on two distinct environments: a gridworld setting and a new text-domain environment (PLUME). PREDICT more accurately infers nuanced human preferences improving over existing baselines by 66.2% (gridworld environment) and 41.0% (PLUME).