# Optimal Interactive Learning on the Job via Facility Location Planning

arXiv: https://arxiv.org/abs/2505.00490 · HTML: https://mlsanity.com/paper/2505.00490/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 37 references that are in the map.
- First posted on arXiv 1 May 2025.

## Builds on (most cited first, 21 of 21)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [robosuite: A Modular Simulation Framework and Benchmark for Robot Learning](https://mlsanity.com/paper/2009.12293/) (2020)
- [Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks](https://mlsanity.com/paper/2306.13831/) (2023)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Reward-rational (implicit) choice: A unifying formalism for reward learning](https://mlsanity.com/paper/2002.04833/) (2020)
- [Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment](https://mlsanity.com/paper/2211.08416/) (2022)
- [Contrastive Preference Learning: Learning from Human Feedback without RL](https://mlsanity.com/paper/2310.13639/) (2023)
- [Learning Reward Functions from Diverse Sources of Human Feedback: Optimally Integrating Demonstrations and Preferences](https://mlsanity.com/paper/2006.14091/) (2020)
- [Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task Parametrizations](https://mlsanity.com/paper/1308.3513/) (2013)
- [Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferences](https://mlsanity.com/paper/2312.09337/) (2023)
- [Unified Learning from Demonstrations, Corrections, and Preferences during Physical Human-Robot Interaction](https://mlsanity.com/paper/2207.03395/) (2022)
- [Conformalized Interactive Imitation Learning: Handling Expert Shift and Intermittent Feedback](https://mlsanity.com/paper/2410.08852/) (2024)
- [Learning Efficient and Robust Language-conditioned Manipulation using Textual-Visual Relevancy and Equivariant Language Mapping](https://mlsanity.com/paper/2406.15677/) (2024)
- [Batch Active Learning of Reward Functions from Human Preferences](https://mlsanity.com/paper/2402.15757/) (2024)
- [Pragmatic Feature Preferences: Learning Reward-Relevant Preferences from Human Input](https://mlsanity.com/paper/2405.14769/) (2024)
- [Learning Human Preferences Over Robot Behavior as Soft Planning Constraints](https://mlsanity.com/paper/2403.19795/) (2024)
- [Synergistic Scheduling of Learning and Allocation of Tasks in Human-Robot Teams](https://mlsanity.com/paper/2203.07478/) (2022)
- [Mapping out the Space of Human Feedback for Reinforcement Learning: A Conceptual Framework](https://mlsanity.com/paper/2411.11761/) (2024)
- [Learning to Optimize Autonomy in Competence-Aware Systems](https://mlsanity.com/paper/2003.07745/) (2020)
- [PECAN: Personalizing Robot Behaviors through a Learned Canonical Space](https://mlsanity.com/paper/2407.16081/) (2024)

## Built on by (most cited first, 2 of 2)

- [Multi-Cycle Spatio-Temporal Adaptation in Human-Robot Teaming](https://mlsanity.com/paper/2604.19670/) (2026)
- [Deliberate Practice: Learning Robot Skills under a Budget](https://mlsanity.com/paper/2608.13415/) (2026)

## Abstract (arXiv)

Collaborative robots must continually adapt to novel tasks and user preferences without overburdening the user. While prior interactive robot learning methods aim to reduce human effort, they are typically limited to single-task scenarios and are not well-suited for sustained, multi-task collaboration. We propose COIL (Cost-Optimal Interactive Learning) -- a multi-task interaction planner that minimizes human effort across a sequence of tasks by strategically selecting among three query types (skill, preference, and help). When user preferences are known, we formulate COIL as an uncapacitated facility location (UFL) problem, which enables bounded-suboptimal planning in polynomial time using off-the-shelf approximation algorithms. We extend our formulation to handle uncertainty in user preferences by incorporating one-step belief space planning, which uses these approximation algorithms as subroutines to maintain polynomial-time performance. Simulated and physical experiments on manipulation tasks show that our framework significantly reduces the amount of work allocated to the human while maintaining successful task completion.