# Steering Generative Robot Policies with Lexicographic Preferences

arXiv: https://arxiv.org/abs/2609.15014 · HTML: https://mlsanity.com/paper/2609.15014/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 14 Sep 2026.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 25 of 25)

- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [Control Barrier Function Based Quadratic Programs for Safety Critical Systems](https://mlsanity.com/paper/1609.06408/) (2016)
- [NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration](https://mlsanity.com/paper/2310.07896/) (2023)
- [Practical and Asymptotically Exact Conditional Sampling in Diffusion Models](https://mlsanity.com/paper/2306.17775/) (2023)
- [A General Framework for Inference-time Scaling and Steering of Diffusion Models](https://mlsanity.com/paper/2501.06848/) (2025)
- [ViNT: A Foundation Model for Visual Navigation](https://mlsanity.com/paper/2306.14846/) (2023)
- [SafeDiffuser: Safe Planning with Diffusion Probabilistic Models](https://mlsanity.com/paper/2306.00148/) (2023)
- [Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrations](https://mlsanity.com/paper/2402.14606/) (2024)
- [NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidance](https://mlsanity.com/paper/2505.08712/) (2025)
- [ViPlanner: Visual Semantic Imperative Learning for Local Navigation](https://mlsanity.com/paper/2310.00982/) (2023)
- [Liability, Ethics, and Culture-Aware Behavior Specification using Rulebooks](https://mlsanity.com/paper/1902.09355/) (2019)
- [Inference-Time Policy Steering through Human Interactions](https://mlsanity.com/paper/2411.16627/) (2024)
- [STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigation](https://mlsanity.com/paper/2103.02828/) (2021)
- [NaviDiffusor: Cost-Guided Diffusion Model for Visual Navigation](https://mlsanity.com/paper/2504.10003/) (2025)
- [VLS: Steering Pretrained Robot Policies via Vision-Language Models](https://mlsanity.com/paper/2602.03973/) (2026)
- [Pareto Navigation Gradient Descent: a First-Order Algorithm for Optimization in Pareto Set](https://mlsanity.com/paper/2110.08713/) (2021)
- [HardFlow: Hard-Constrained Sampling for Flow-Matching Models via Trajectory Optimization](https://mlsanity.com/paper/2511.08425/) (2025)
- [Shield Model Predictive Path Integral: A Computationally Efficient Robust MPC Approach Using Control Barrier Functions](https://mlsanity.com/paper/2302.11719/) (2023)
- [SEAM: Smooth Execution of Action-Chunked Motion for Vision-Language-Action Policies](https://mlsanity.com/paper/2607.04609/) (2026)
- [Sem-NaVAE: Semantically-Guided Outdoor Mapless Navigation via Generative Trajectory Priors](https://mlsanity.com/paper/2602.01429/) (2026)

## Built on by (most cited first, 1 of 1)

- [PreferenceFlow: Test-Time Guidance of Flow-Matching Robot Policies from Human Interventions](https://mlsanity.com/paper/2609.36872/) (2026)

## Abstract (arXiv)

Pretrained generative robot policies can produce effective behaviors across diverse environments, but deployment can lead to requirements and preferences that may not have been represented during training. Furthermore, at deployment, an operator, user, or application may assign these requirements and preferences a priority order that can vary across deployments. For example, embodiment-specific feasibility constraints may need to be satisfied first, while user-specific preferences guide behavior among the feasible options. We show that a frozen generative robot policy---based on either diffusion or flow matching---can be steered at inference time to respect such lexicographically ordered deployment objectives. To achieve this, we introduce two modifications to the sampler. First, we apply dynamic-barrier guidance to sampled trajectories, constraining lower-priority updates so that higher-priority costs do not increase (up to first order). Second, we select the executed sample using a cascade that successively filters candidate samples according to each priority level. The policy weights remain unchanged. On a navigation benchmark, we demonstrate that our method improves success, traversability, and preference compliance over the frozen policy, and achieves substantially better compliance than tuned weighted-sum baselines. The same method transfers to a flow-matching manipulation policy on LIBERO, where it improves compliance without reducing task success. A controlled manipulation study further shows that, in settings where a fixed weight can match the desired ordering, the dynamic barrier reaches comparable best performance over a substantially wider range of parameter settings.