mlsanityData to 7 Oct 2026

LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learning

arXiv 2209.10341 ↗First posted 21 Sep 2022cs.LG cs.AI cs.LOcited by 11

Not on the map. Why?

Abstract (arXiv)

LCRL is a software tool that implements model-free Reinforcement Learning (RL) algorithms over unknown Markov Decision Processes (MDPs), synthesising policies that satisfy a given linear temporal specification with maximal probability. LCRL leverages partially deterministic finite-state machines known as Limit Deterministic Buchi Automata (LDBA) to express a given linear temporal specification. A reward function for the RL algorithm is shaped on-the-fly, based on the structure of the LDBA. Theoretical guarantees under proper assumptions ensure the convergence of the RL algorithm to an optimal policy that maximises the satisfaction probability. We present case studies to demonstrate the applicability, ease of use, scalability, and performance of LCRL. Owing to the LDBA-guided exploration and LCRL model-free architecture, we observe robust performance, which also scales well when compared to standard RL approaches (whenever applicable to LTL specifications). Full instructions on how to execute all the case studies in this paper are provided on a GitHub page that accompanies the LCRL distribution www.github.com/grockious/lcrl.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 21 of its 46 references that are in the map. First posted on arXiv 21 Sep 2022.

Builds on

21 of 46 references.

References · 21

  1. 2018DeepMind Control Suitecited by 935
  2. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  3. 2019Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guaranteescited by 77
  4. 2019Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learningcited by 74
  5. 2018Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specificationscited by 69
  6. 2020A Composable Specification Language for Reinforcement Learning Taskscited by 68
  7. 2020The Probabilistic Model Checker Stormcited by 64
  8. 2018Logically-Constrained Reinforcement Learningcited by 50
  9. 2014A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specificationscited by 49
  10. 2020Cautious Reinforcement Learning with Logical Constraintscited by 44
  11. 2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logiccited by 38
  12. 2019Certified Reinforcement Learning with Logic Guidancecited by 35
9 more
  1. 2019Rethinking the Discount Factor in Reinforcement Learning: A Decision Theoretic Approachcited by 32
  2. 2020Verifiably Safe Exploration for End-to-End Reinforcement Learningcited by 32
  3. 2022COOL-MC: A Comprehensive Tool for Reinforcement Learning and Model Checkingcited by 19
  4. 2018Logically-Constrained Neural Fitted Q-Iterationcited by 17
  5. 2019Verifiably Safe Off-Model Reinforcement Learningcited by 16
  6. 2014FAUST2: Formal Abstractions of Uncountable-STate STochastic processescited by 12
  7. 2014Formal Consistency Checking over Specifications in Natural Languagescited by 8
  8. 2019DeepSynth: Automata Synthesis for Automatic Task Segmentation in Deep Reinforcement Learningcited by 8
  9. 2021Mungojerrie: Reinforcement Learning of Linear-Time Objectivescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 1; 2025: 2; 2026: 2.

11 papers.

  1. 2023ALLURE: Auditing and Improving LLM-based Evaluation of Text using Iterative In-Context-Learningcited by 15
  2. 2023Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specificationscited by 3
  3. 2024Sample-Efficient Reinforcement Learning with Temporal Logic Objectives: Leveraging the Task Specification to Guide Explorationcited by 3
  4. 2023System III: Learning with Domain Knowledge for Safety Constraintscited by 2
  5. 2026Semantically Labelled Automata for Multi-Task Reinforcement Learning with LTL Instructionscited by 2
  6. 2023Co-learning Planning and Control Policies Constrained by Differentiable Logic Specificationscited by 1
  7. 2023Probabilistic Counterexample Guidance for Safer Reinforcement Learning (Extended Version)cited by 0
  8. 2023Verified Compositional Neuro-Symbolic Control for Stochastic Systems with Temporal Logic Taskscited by 0
  9. 2025Autonomous Learning From Success and Failure: Goal-Conditioned Supervised Learning with Negative Feedbackcited by 0
  10. 2025Automating the Refinement of Reinforcement Learning Specificationscited by 0
  11. 2026Value Functions as Supermartingale Certificatescited by 0