Signal Temporal Logic-Guided Apprenticeship Learning
Not on the map. Why?
Abstract (arXiv)
Apprenticeship learning crucially depends on effectively learning rewards, and hence control policies from user demonstrations. Of particular difficulty is the setting where the desired task consists of a number of sub-goals with temporal dependencies. The quality of inferred rewards and hence policies are typically limited by the quality of demonstrations, and poor inference of these can lead to undesirable outcomes. In this letter, we show how temporal logic specifications that describe high level task objectives, are encoded in a graph to define a temporal-based metric that reasons about behaviors of demonstrators and the learner agent to improve the quality of inferred rewards and policies. Through experiments on a diverse set of robot manipulator simulations, we show how our framework overcomes the drawbacks of prior literature by drastically improving the number of demonstrations required to learn a control policy.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 29 of its 51 references that are in the map. First posted on arXiv 9 Nov 2023.
Builds on
29 of 51 references.
References · 29
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2017Hindsight Experience Replaycited by 1,096
- 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
- 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
- 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
- 2018Behavioral Cloning from Observationcited by 482
- 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
- 2018Generative Adversarial Imitation from Observationcited by 183
- 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 183
- 2019Goal-conditioned Imitation Learningcited by 159
- 2020Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Criticscited by 154
17 more
- 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 135
- 2017Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Controlcited by 92
- 2020Learning from Suboptimal Demonstration via Self-Supervised Reward Regressioncited by 84
- 2021On the Expressivity of Markov Rewardcited by 71
- 2022Watch and Match: Supercharging Imitation with Regularized Optimal Transportcited by 64
- 2021panda-gym: Open-source goal-conditioned environments for robotic learningcited by 56
- 2021SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learningcited by 28
- 2023Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robotcited by 21
- 2020RTAMT: Online Robustness Monitors from STLcited by 19
- 2020Active Task-Inference-Guided Deep Inverse Reinforcement Learningcited by 17
- 2021Learning from Demonstrations using Signal Temporal Logiccited by 17
- 2019Control from Signal Temporal Logic Specifications with Smooth Cumulative Quantitative Semanticscited by 16
- 2021OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matchingcited by 15
- 2022Monte Carlo Augmented Actor-Critic for Sparse Reward Deep Reinforcement Learning from Suboptimal Demonstrationscited by 14
- 2017Safety-Aware Apprenticeship Learningcited by 12
- 2018Automata Guided Reinforcement Learning With Demonstrationscited by 5
- 2022Learning Performance Graphs from Demonstrations via Task-Based Evaluationscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.