Omega-Regular Reward Machines
Not on the map. Why?
Abstract (arXiv)
Reinforcement learning (RL) is a powerful approach for training agents to perform tasks, but designing an appropriate reward mechanism is critical to its success. However, in many cases, the complexity of the learning objectives goes beyond the capabilities of the Markovian assumption, necessitating a more sophisticated reward mechanism. Reward machines and omega-regular languages are two formalisms used to express non-Markovian rewards for quantitative and qualitative objectives, respectively. This paper introduces omega-regular reward machines, which integrate reward machines with omega-regular languages to enable an expressive and effective reward mechanism for RL. We present a model-free RL algorithm to compute epsilon-optimal strategies against omega-egular reward machines and evaluate the effectiveness of the proposed algorithm through experiments.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 9 of its 26 references that are in the map. First posted on arXiv 14 Aug 2023.
Builds on
9 of 26 references.
References · 9
- 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 191
- 2019Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guaranteescited by 77
- 2018Omega-Regular Objectives in Model-Free Reinforcement Learningcited by 75
- 2019Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learningcited by 75
- 2014A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specificationscited by 49
- 2019Reinforcement Learning with Non-Markovian Rewardscited by 45
- 2022Lexicographic Multi-Objective Reinforcement Learningcited by 17
- 2020Reinforcement Learning of Control Policy for Linear Temporal Logic Specifications Using Limit-Deterministic Generalized B"uchi Automatacited by 11
- 2021Model-Free Learning of Safe yet Effective Controllerscited by 5
Built on by
Citing papers per year: 2025: 2.
2 papers.