mlsanityData to 7 Oct 2026

Maximally Permissive Reward Machines

arXiv 2408.08059 ↗First posted 15 Aug 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Reward machines allow the definition of rewards for temporally extended tasks and behaviors. Specifying "informative" reward machines can be challenging. One way to address this is to generate reward machines from a high-level abstract description of the learning environment, using techniques such as AI planning. However, previous planning-based approaches generate a reward machine based on a single (sequential or partial-order) plan, and do not allow maximum flexibility to the learning agent. In this paper we propose a new approach to synthesising reward machines which is based on the set of partial order plans for a goal. We prove that learning using such "maximally permissive" reward machines results in higher rewards than learning using RMs based on a single plan. We present experimental results which support our theoretical claims by showing that our approach obtains higher rewards than the single-plan approach in practice.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 11 of its 28 references that are in the map. First posted on arXiv 15 Aug 2024.

Builds on

11 of 28 references.

References · 11

  1. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  2. 2024Reinforcement learningcited by 236
  3. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  4. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 183
  5. 2018SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planningcited by 77
  6. 2019Joint Inference of Reward Machines and Policies for Reinforcement Learningcited by 56
  7. 2018PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Makingcited by 51
  8. 2020Reward Machines for Cooperative Multi-Agent Reinforcement Learningcited by 35
  9. 2023Plan To Predict: Learning an Uncertainty-Foreseeing Model for Model-Based Reinforcement Learningcited by 16
  10. 2021Inferring Probabilistic Reward Machines from Non-Markovian Reward Processes for Reinforcement Learningcited by 13
  11. 2021Decentralized Graph-Based Multi-Agent Reinforcement Learning Using Reward Machinescited by 6

Built on by

Not cited yet.