mlsanityData to 14 Oct 2026

Imitation Learning via Focused Satisficing

arXiv 2505.14820 ↗First posted 20 May 2025cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning often assumes that demonstrations are close to optimal according to some fixed, but unknown, cost function. However, according to satisficing theory, humans often choose acceptable behavior based on their personal (and potentially dynamic) levels of aspiration, rather than achieving (near-) optimality. For example, a lunar lander demonstration that successfully lands without crashing might be acceptable to a novice despite being slow or jerky. Using a margin-based objective to guide deep reinforcement learning, our focused satisficing approach to imitation learning seeks a policy that surpasses the demonstrator's aspiration levels -- defined over trajectories or portions of trajectories -- on unseen demonstrations without explicitly learning those aspirations. We show experimentally that this focuses the policy to imitate the highest quality (portions of) demonstrations better than existing imitation learning methods, providing much higher rates of guaranteed acceptability to the demonstrator, and competitive true returns on a range of environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 13 of its 33 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

13 of 33 references.

Closest ancestors · 4

Same direction, newest first

  1. 2021Confidence-Aware Imitation Learning from Demonstrations with Varying Optimality27 Oct
  2. 2020Robust Imitation Learning from Noisy Demonstrations20 Oct
  3. 2019Imitation Learning from Imperfect Demonstration27 Jan
  4. 2018An Algorithmic Perspective on Imitation Learning16 Nov

Other references · 9

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2017Deep reinforcement learning from human preferencescited by 3,912
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  5. 2016Concrete Problems in AI Safetycited by 1,642
  6. 2016Generative Adversarial Imitation Learningcited by 1,597
  7. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 260
  8. 2020Learning from Suboptimal Demonstration via Self-Supervised Reward Regressioncited by 84
  9. 2023Superhuman Fairnesscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges
  2. 2025Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Exploration
  3. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances
  4. 2022Towards Improving Exploration in Self-Imitation Learning using Intrinsic Motivation
  5. 2019Better-than-Demonstrator Imitation Learning via Automatically-Ranked Demonstrations
  6. 2021Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learning
  7. 2021Imitation Learning: Progress, Taxonomies and Challenges
  8. 2023Skill Disentanglement for Imitation Learning from Suboptimal Demonstrations