Data to 14 Oct 2026

Omega-Regular Objectives in Model-Free Reinforcement Learning

arXiv 1810.00950 ↗First posted 26 Sep 2018cs.LO cs.LG stat.MLcited by 75

Not on the map. Why?

Cited by 27 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We provide the first solution for model-free reinforcement learning of ω-regular objectives for Markov decision processes (MDPs). We present a constructive reduction from the almost-sure satisfaction of ω-regular objectives to an almost- sure reachability problem and extend this technique to learning how to control an unknown model so that the chance of satisfying the objective is maximized. A key feature of our technique is the compilation of ω-regular properties into limit- deterministic Buechi automata instead of the traditional Rabin automata; this choice sidesteps difficulties that have marred previous proposals. Our approach allows us to apply model-free, off-the-shelf reinforcement learning algorithms to compute optimal strategies from the observations of the MDP. We present an experimental evaluation of our technique on benchmark learning problems.

Key facts. Cited by 75 papers in the map (data through 14 Oct 2026). First posted on arXiv 26 Sep 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 12; 2022: 11; 2023: 14; 2024: 13; 2025: 11; 2026: 14.

75 papers.

Sort:

By area:

2021Compositional Reinforcement Learning from Logical Specificationscited by 67
2021Safe Multi-Agent Reinforcement Learning via Shieldingcited by 61
2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logiccited by 39
2021How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Reviewcited by 39
2022Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarkingcited by 34
2021Multi-Agent Reinforcement Learning with Temporal Logic Specificationscited by 27
2022Online Shielding for Reinforcement Learningcited by 22
2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 21
2022COOL-MC: A Comprehensive Tool for Reinforcement Learning and Model Checkingcited by 19
2021On the (In)Tractability of Reinforcement Learning for LTL Objectivescited by 18
2023Sample Efficient Model-free Reinforcement Learning from LTL Specifications with Optimality Guaranteescited by 18
2023Policy Synthesis and Reinforcement Learning for Discounted LTLcited by 14

38 more

2023Compositional Policy Learning in Stochastic Control Systems with Formal Guaranteescited by 14
2024Reinforcement Learning with LTL and ω-Regular Objectives via Optimality-Preserving Translation to Average Rewardscited by 14
2021Verifiable and Compositional Reinforcement Learning Systemscited by 13
2021Inferring Probabilistic Reward Machines from Non-Markovian Reward Processes for Reinforcement Learningcited by 13
2023Computably Continuous Reinforcement-Learning Objectives are PAC-learnablecited by 13
2022Joint Learning of Reward Machines and Policies in Environments with Partially Known Semanticscited by 12
2023Model-based Dynamic Shielding for Safe and Efficient Multi-Agent Reinforcement Learningcited by 11
2022Accelerated Reinforcement Learning for Temporal Logic Control Objectivescited by 9
2024Safety-Oriented Pruning and Interpretation of Reinforcement Learning Policiescited by 9
2022Specification-Guided Learning of Nash Equilibria with High Social Welfarecited by 8
2021Mungojerrie: Reinforcement Learning of Linear-Time Objectivescited by 6
2022Model-Free Reinforcement Learning for Symbolic Automata-encoded Objectivescited by 6
2024Enhancing RL Safety with Counterfactual LLM Reasoningcited by 6
2021Model-Free Learning of Safe yet Effective Controllerscited by 5
2022Targeted Adversarial Attacks on Deep Reinforcement Learning Policies via Model Checkingcited by 5
2024Probabilistic Model Checking of Stochastic Reinforcement Learning Policiescited by 5
2024What Are the Odds? Improving the foundations of Statistical Model Checkingcited by 5
2024Sample-Efficient Reinforcement Learning with Temporal Logic Objectives: Leveraging the Task Specification to Guide Explorationcited by 5
20241-2-3-Go! Policy Synthesis for Parameterized Markov Decision Processes via Decision-Tree Learning and Generalizationcited by 5
2022PAC Statistical Model Checking of Mean Payoff in Discrete- and Continuous-Time MDPcited by 4
2021Learning Optimal Strategies for Temporal Tasks in Stochastic Gamescited by 3
2023Verifiable Reinforcement Learning Systems via Compositionalitycited by 3
2023Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specificationscited by 3
2024On the Uniqueness of Solution for the Bellman Equation of LTL Objectivescited by 3
2024Inductive Generalization in Reinforcement Learning from Specificationscited by 3
2024Tractable Offline Learning of Regular Decision Processescited by 3
2025Turn-based Multi-Agent Reinforcement Learning Model Checkingcited by 3
2021Reinforcement Learning Based Temporal Logic Control with Soft Constraints Using Limit-deterministic Generalized Buchi Automatacited by 2
2023Optimal Control of Logically Constrained Partially Observable and Multi-Agent Markov Decision Processescited by 2
2023Omega-Regular Reward Machinescited by 2
2025Co-Activation Graph Analysis of Safety-Verified and Explainable Deep Reinforcement Learning Policiescited by 2
2025Automaton Constrained Q-Learningcited by 2
2026PlatoLTL: Scaling LTL-Guided Multi-Task RLcited by 2
2026Semantically Labelled Automata for Multi-Task Reinforcement Learning with LTL Instructionscited by 2
2023Reinforcement Learning for Omega-Regular Specifications on Continuous-Time MDPcited by 1
2023Model-free Motion Planning of Autonomous Agents for Complex Tasks in Partially Observable Environmentscited by 1
2024Integrating Explanations in Learning LTL Specifications from Demonstrationscited by 1
2024Regret-Free Reinforcement Learning for LTL Specificationscited by 1

50 most cited of 75 All 75 (JSON)