# Multi-Objective Planning with Contextual Lexicographic Reward Preferences

arXiv: https://arxiv.org/abs/2502.10476 · HTML: https://mlsanity.com/paper/2502.10476/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 34 references that are in the map.
- First posted on arXiv 13 Feb 2025.

## Builds on (most cited first, 11 of 11)

- [Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences](https://mlsanity.com/paper/2002.09089/) (2020)
- [Online Bayesian Goal Inference for Boundedly-Rational Planning Agents](https://mlsanity.com/paper/2006.07532/) (2020)
- [Markov Decision Processes with Continuous Side Information](https://mlsanity.com/paper/1711.05726/) (2017)
- [Information Retrieval: Recent Advances and Beyond](https://mlsanity.com/paper/2301.08801/) (2023)
- [Contextualize Me -- The Case for Context in Reinforcement Learning](https://mlsanity.com/paper/2202.04500/) (2022)
- [Uncertainty-Aware Search Framework for Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2204.05944/) (2022)
- [Robust Multi-Objective Bayesian Optimization Under Input Noise](https://mlsanity.com/paper/2202.07549/) (2022)
- [Context-Aware Planning and Environment-Aware Memory for Instruction Following Embodied Agents](https://mlsanity.com/paper/2308.07241/) (2023)
- [Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges](https://mlsanity.com/paper/2403.00784/) (2024)
- [Scalarizing Multi-Objective Robot Planning Problems using Weighted Maximization](https://mlsanity.com/paper/2312.07227/) (2023)
- [Context-Aware Route Planning for Automated Warehouses](https://mlsanity.com/paper/1901.07422/) (2019)

## Abstract (arXiv)

Autonomous agents are often required to plan under multiple objectives whose preference ordering varies based on context. The agent may encounter multiple contexts during its course of operation, each imposing a distinct lexicographic ordering over the objectives, with potentially different reward functions associated with each context. Existing approaches to multi-objective planning typically consider a single preference ordering over the objectives, across the state space, and do not support planning under multiple objective orderings within an environment. We present Contextual Lexicographic Markov Decision Process (CLMDP), a framework that enables planning under varying lexicographic objective orderings, depending on the context. In a CLMDP, both the objective ordering at a state and the associated reward functions are determined by the context. We employ a Bayesian approach to infer a state-context mapping from expert trajectories. Our algorithm to solve a CLMDP first computes a policy for each objective ordering and then combines them into a single context-aware policy that is valid and cycle-free. The effectiveness of the proposed approach is evaluated in simulation and using a mobile robot.