mlsanityData to 14 Oct 2026

Learning Performance Graphs from Demonstrations via Task-Based Evaluations

arXiv 2204.05909 ↗First posted 12 Apr 2022cs.RO cs.LGcited by 3

On the map: Offline RL and imitation learning from data → Robot learning from demonstration

Abstract (arXiv)

In the learning from demonstration (LfD) paradigm, understanding and evaluating the demonstrated behaviors plays a critical role in extracting control policies for robots. Without this knowledge, a robot may infer incorrect reward functions that lead to undesirable or unsafe control policies. Recent work has proposed an LfD framework where a user provides a set of formal task specifications to guide LfD, to address the challenge of reward shaping. However, in this framework, specifications are manually ordered in a performance graph (a partial order that specifies relative importance between the specifications). The main contribution of this paper is an algorithm to learn the performance graph directly from the user-provided demonstrations, and show that the reward functions generated using the learned performance graph generate similar policies to those from manually specified performance graphs. We perform a user study that shows that priorities specified by users on behaviors in a simulated highway driving domain match the automatically inferred performance graph. This establishes that we can accurately evaluate user demonstrations with respect to task specifications without expert criteria.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data. Builds on 24 of its 62 references that are in the map. First posted on arXiv 12 Apr 2022.

Builds on

24 of 62 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Learning from Demonstrations using Signal Temporal Logic15 Feb

Other references · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  3. 2016Concrete Problems in AI Safetycited by 1,642
  4. 2018Behavioral Cloning from Observationcited by 492
  5. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  6. 2018Distributed Distributional Deterministic Policy Gradientscited by 199
  7. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  8. 2016Reinforcement Learning With Temporal Logic Rewardscited by 117
  9. 2019Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspectivecited by 108
  10. 2020Learning Reward Functions from Diverse Sources of Human Feedback: Optimally Integrating Demonstrations and Preferencescited by 71
  11. 2021On the Expressivity of Markov Rewardcited by 71
  12. 2016Q-Learning for Robust Satisfaction of Signal Temporal Logic Specificationscited by 67
11 more
  1. 2020RTAMT: Online Robustness Monitors from STLcited by 19
  2. 2020Active Task-Inference-Guided Deep Inverse Reinforcement Learningcited by 17
  3. 2017Safety-Aware Apprenticeship Learningcited by 12
  4. 2015Correct-by-synthesis reinforcement learning with temporal logic constraintscited by 11
  5. 2020Dynamic Multi-Robot Task Allocation under Uncertainty and Temporal Constraintscited by 9
  6. 2020robo-gym -- An Open Source Toolkit for Distributed Deep Reinforcement Learning on Real and Simulated Robotscited by 9
  7. 2020Event-Based Signal Temporal Logic Synthesis for Single and Multi-Robot Taskscited by 7
  8. 2020Formalizing and Guaranteeing* Human-Robot Interactioncited by 6
  9. 2018Automata Guided Reinforcement Learning With Demonstrationscited by 5
  10. 2021Study of Signal Temporal Logic Robustness Metrics for Robotic Tasks Optimizationcited by 4
  11. 2021Explaining Reward Functions to Humans for Better Human-Robot Collaborationcited by 3

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1.

3 papers.

  1. 2023A Safe Preference Learning Approach for Personalization with Applications to Autonomous Vehiclescited by 1
  2. 2023Signal Temporal Logic-Guided Apprenticeship Learningcited by 1
  3. 2024Scaling Learning based Policy Optimization for Temporal Logic Tasks by Controller Network Dropoutcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Reinforcement learning with Demonstrations from Mismatched Task under Sparse Reward
  2. 2022Quantifying Demonstration Quality for Robot Learning and Generalization
  3. 2024Reward Machine Inference for Robotic Manipulation
  4. 2019Benchmark for Skill Learning from Demonstration: Impact of User Experience, Task Complexity, and Start Configuration on Performance
  5. 2025Demonstration Sidetracks: Categorizing Systematic Non-Optimality in Human Demonstrations
  6. 2022Autonomous Assessment of Demonstration Sufficiency via Bayesian Inverse Reinforcement Learning
  7. 2026Learning from Imperfect Demonstrations via Temporal Behavior Tree-Guided Trajectory Repair
  8. 2021Ergodic imitation: Learning from what to do and what not to do