mlsanityData to 14 Oct 2026

Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learning

arXiv 1909.07299 ↗First posted 16 Sep 2019cs.RO cs.AI cs.LGcited by 75

Not on the map. Why?

Cited by 27 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a reinforcement learning (RL) framework to synthesize a control policy from a given linear temporal logic (LTL) specification in an unknown stochastic environment that can be modeled as a Markov Decision Process (MDP). Specifically, we learn a policy that maximizes the probability of satisfying the LTL formula without learning the transition probabilities. We introduce a novel rewarding and path-dependent discounting mechanism based on the LTL formula such that (i) an optimal policy maximizing the total discounted reward effectively maximizes the probabilities of satisfying LTL objectives, and (ii) a model-free RL algorithm using these rewards and discount factors is guaranteed to converge to such policy. Finally, we illustrate the applicability of our RL-based synthesis approach on two motion planning case studies.

Key facts. Cited by 75 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 12; 2022: 8; 2023: 19; 2024: 14; 2025: 9; 2026: 13.

75 papers.

Sort:

By area:

  1. 2021Safe Multi-Agent Reinforcement Learning via Shieldingcited by 61
  2. 2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logiccited by 39
  3. 2021Multi-Agent Reinforcement Learning with Temporal Logic Specificationscited by 27
  4. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 21
  5. 2022Policy Optimization with Linear Temporal Logic Constraintscited by 19
  6. 2023Eventual Discounting Temporal Logic Counterfactual Experience Replaycited by 19
  7. 2024Compositional Automata Embeddings for Goal-Conditioned Reinforcement Learningcited by 19
  8. 2021On the (In)Tractability of Reinforcement Learning for LTL Objectivescited by 18
  9. 2023Sample Efficient Model-free Reinforcement Learning from LTL Specifications with Optimality Guaranteescited by 18
  10. 2023Policy Synthesis and Reinforcement Learning for Discounted LTLcited by 14
  11. 2023Compositional Policy Learning in Stochastic Control Systems with Formal Guaranteescited by 14
  12. 2024Reinforcement Learning with LTL and ω-Regular Objectives via Optimality-Preserving Translation to Average Rewardscited by 14
38 more
  1. 2023Computably Continuous Reinforcement-Learning Objectives are PAC-learnablecited by 13
  2. 2023A PAC Learning Algorithm for LTL and Omega-regular Objectives in MDPscited by 13
  3. 2022Joint Learning of Reward Machines and Policies in Environments with Partially Known Semanticscited by 12
  4. 2022LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learningcited by 11
  5. 2023Model-based Dynamic Shielding for Safe and Efficient Multi-Agent Reinforcement Learningcited by 11
  6. 2023LgTS: Dynamic Task Sampling using LLM-generated sub-goals for Reinforcement Learning Agentscited by 11
  7. 2024LTL-Constrained Policy Optimization with Cycle Experience Replaycited by 11
  8. 2024Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learningcited by 10
  9. 2022Accelerated Reinforcement Learning for Temporal Logic Control Objectivescited by 9
  10. 2024Directed Exploration in Reinforcement Learning from Linear Temporal Logiccited by 7
  11. 2021Mungojerrie: Reinforcement Learning of Linear-Time Objectivescited by 6
  12. 2024Adaptive Reward Design for Reinforcement Learningcited by 6
  13. 2021Model-Free Learning of Safe yet Effective Controllerscited by 5
  14. 2021Lifelong Reinforcement Learning with Temporal Logic Formulas and Reward Machinescited by 5
  15. 2023Reinforcement Learning with Knowledge Representation and Reasoning: A Brief Surveycited by 5
  16. 2024Sample-Efficient Reinforcement Learning with Temporal Logic Objectives: Leveraging the Task Specification to Guide Explorationcited by 5
  17. 2021Learning Optimal Strategies for Temporal Tasks in Stochastic Gamescited by 3
  18. 2022Logic-based Reward Shaping for Multi-Agent Reinforcement Learningcited by 3
  19. 2023Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specificationscited by 3
  20. 2024Logical Specifications-guided Dynamic Task Sampling for Reinforcement Learning Agentscited by 3
  21. 2024On the Uniqueness of Solution for the Bellman Equation of LTL Objectivescited by 3
  22. 2024Tractable Offline Learning of Regular Decision Processescited by 3
  23. 2021Reinforcement Learning Based Temporal Logic Control with Soft Constraints Using Limit-deterministic Generalized Buchi Automatacited by 2
  24. 2022SAFER: Safe Collision Avoidance using Focused and Efficient Trajectory Search with Reinforcement Learningcited by 2
  25. 2023Co-learning Planning and Control Policies Constrained by Differentiable Logic Specificationscited by 2
  26. 2023Optimal Control of Logically Constrained Partially Observable and Multi-Agent Markov Decision Processescited by 2
  27. 2023Omega-Regular Reward Machinescited by 2
  28. 2025Learning Probabilistic Temporal Logic Specifications for Stochastic Systemscited by 2
  29. 2026PlatoLTL: Scaling LTL-Guided Multi-Task RLcited by 2
  30. 2026Semantically Labelled Automata for Multi-Task Reinforcement Learning with LTL Instructionscited by 2
  31. 2021Safety-Critical Learning of Robot Control with Temporal Logic Specificationscited by 1
  32. 2021Learning Spatio-Temporal Specifications for Dynamical Systemscited by 1
  33. 2022Synthesis from Satisficing and Temporal Goalscited by 1
  34. 2023Reinforcement Learning for Omega-Regular Specifications on Continuous-Time MDPcited by 1
  35. 2023Model-free Motion Planning of Autonomous Agents for Complex Tasks in Partially Observable Environmentscited by 1
  36. 2024Learning Hidden Subgoals under Temporal Ordering Constraints in Reinforcement Learningcited by 1
  37. 2025Accelerated Learning with Linear Temporal Logic using Differentiable Simulationcited by 1
  38. 2026Zero-Shot Instruction Following in RL via Structured LTL Representationscited by 1

50 most cited of 75 All 75 (JSON)