mlsanityData to 14 Oct 2026

Split the Differences, Pool the Rest: Provably Efficient Multi-Objective Imitation

arXiv 2605.12000 ↗First posted 12 May 2026cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

This work investigates multi-objective imitation learning: the problem of recovering policies that lie on the Pareto front given demonstrations from multiple Pareto-optimal experts in a Multi-Objective Markov Decision Process (MOMDP). Standard imitation approaches are ill-equipped for this regime, as naively aggregating conflicting expert trajectories can result in dominated policies. To address this, we introduce Multi-Output Augmented Behavioral Cloning (MA-BC), an algorithm that systematically partitions divergent expert data while pooling state-action pairs where no behavior conflict is observed. Theoretically, we prove that MA-BC converges to Pareto-optimal policies at a faster statistical rate than any learner that considers each expert dataset independently. Furthermore, we establish a novel lower bound for multi-objective imitation learning, demonstrating that MA-BC is minimax optimal. Finally, we empirically validate our algorithm across diverse discrete environments and, guided by our theoretical insights, extend and evaluate MA-BC on a continuous Linear Quadratic Regulator (LQR) control task.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 41 of its 65 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

41 of 65 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026Multi-agent imitation learning with function approximation: Linear Markov games and beyond26 Feb
  2. 2025Rate optimal learning of equilibria from data10 Oct
  3. 2025Inverse Q-Learning Done Right: Offline Imitation Learning in Qπ-Realizable MDPs26 May
  4. 2025Learning Equilibria from Data: Provably Efficient Multi-Agent Imitation Learning23 May
  5. 2024Learning Strategy Representation for Imitation Learning in Multi-Agent Games28 Sep
  6. 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning20 Jul
  7. 2024Multi-Agent Imitation Learning: Value is Easy, Regret is Hard6 Jun
  8. 2024Imitation Learning in Discounted Linear MDPs without exploration assumptions3 May
  9. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions11 Jun
  10. 2022Proximal Point Imitation Learning22 Sep
  11. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap4 Mar
  12. 2020Toward the Fundamental Limits of Imitation Learning13 Sep
2 more
  1. 2018An Algorithmic Perspective on Imitation Learning16 Nov
  2. 2018Multi-Agent Generative Adversarial Imitation Learning26 Jul

Other references · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  5. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  6. 2016Generative Adversarial Imitation Learningcited by 1,597
  7. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  8. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  9. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  10. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
  11. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 406
  12. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 380
15 more
  1. 2023Pretraining Language Models with Human Preferencescited by 247
  2. 2014A Survey of Multi-Objective Sequential Decision-Makingcited by 242
  3. 2019A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptationcited by 177
  4. 2024Behavior Generation with Latent Actionscited by 160
  5. 2017Coordinated Multi-Agent Imitation Learningcited by 72
  6. 2019Multi-Agent Adversarial Inverse Reinforcement Learningcited by 60
  7. 2016Model-Free Imitation Learning with Policy Optimizationcited by 52
  8. 2021Online Apprenticeship Learningcited by 29
  9. 2024Imitating Language via Scalable Inverse Reinforcement Learningcited by 21
  10. 2025Optimistically Optimistic Exploration for Provably Efficient Infinite-Horizon Reinforcement and Imitation Learningcited by 12
  11. 2024How to Find the Exact Pareto Front for Multi-Objective MDPs?cited by 5
  12. 2025Quantization-Free Autoregressive Action Transformercited by 5
  13. 2024Select to Perfect: Imitating desired behavior from large multi-agent datacited by 4
  14. 2024Multi-Objective LQR with Linear Scalarizationcited by 2
  15. 2024Pareto Inverse Reinforcement Learning for Diverse Expert Policy Generationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022TaSIL: Taylor Series Imitation Learning
  2. 2019A Divergence Minimization Perspective on Imitation Learning Methods
  3. 2022Imitation Learning by Estimating Expertise of Demonstrators
  4. 2025The Pitfalls of Imitation Learning when Actions are Continuous
  5. 2022Data augmentation for efficient learning from parametric experts
  6. 2026Difference-Aware Retrieval Policies for Imitation Learning
  7. 2023Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior
  8. 2024Imitation Learning from Suboptimal Demonstrations via Meta-Learning An Action Ranker