mlsanityData to 7 Oct 2026

Active Reward Machine Inference From Raw State Trajectories

arXiv 2604.07480 ↗First posted 8 Apr 2026cs.RO cs.AI cs.FLcited by 0

Not on the map. Why?

Abstract (arXiv)

Reward machines are automaton-like structures that capture the memory required to accomplish a multi-stage task. When combined with reinforcement learning or optimal control methods, they can be used to synthesize robot policies to achieve such tasks. However, specifying a reward machine by hand, including a labeling function capturing high-level features that the decisions are based on, can be a daunting task. This paper deals with the problem of learning reward machines directly from raw state and policy information. As opposed to existing works, we assume no access to observations of rewards, labels, or machine nodes, and show what trajectory data is sufficient for learning the reward machine in this information-scarce regime. We then extend the result to an active learning setting where we incrementally query trajectory extensions to improve data (and indirectly computational) efficiency. Results are demonstrated with several grid world examples.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 38 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

21 of 38 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Concrete Problems in AI Safetycited by 1,582
  3. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  4. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 183
  5. 2018Batch Active Preference-Based Learning of Reward Functionscited by 87
  6. 2019Asking Easy Questions: A User-Friendly Approach to Active Reward Learningcited by 67
  7. 2019Joint Inference of Reward Machines and Policies for Reinforcement Learningcited by 56
  8. 2021Identifiability in inverse reinforcement learningcited by 55
  9. 2020Active Finite Reward Automaton Inference and Reinforcement Learning Using Queries and Counterexamplescited by 23
  10. 2019Induction of Subgoal Automata for Reinforcement Learningcited by 17
  11. 2020Active Task-Inference-Guided Deep Inverse Reinforcement Learningcited by 17
  12. 2021Learning Reward Machines: A Study in Partially Observable Reinforcement Learningcited by 16
9 more
  1. 2022Joint Learning of Reward Machines and Policies in Environments with Partially Known Semanticscited by 12
  2. 2024Reward Machines for Deep RL in Noisy and Uncertain Environmentscited by 11
  3. 2022Learning Task Automata for Reinforcement Learning using Hidden Markov Modelscited by 7
  4. 2024Learning Robust Reward Machines from Noisy Labelscited by 7
  5. 2021Learning Quadruped Locomotion Policies using Logical Rulescited by 6
  6. 2023A Mathematical Characterization of Minimally Sufficient Robot Brainscited by 4
  7. 2024Reward Machine Inference for Robotic Manipulationcited by 4
  8. 2025Learning Reward Machines from Partially Observed Policiescited by 3
  9. 2026Toward Learning POMDPs Beyond Full-Rank Actions and State Observabilitycited by 1

Built on by

Not cited yet.