mlsanityData to 7 Oct 2026

Accelerated Reinforcement Learning for Temporal Logic Control Objectives

arXiv 2205.04424 ↗First posted 9 May 2022cs.RO cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

This paper addresses the problem of learning control policies for mobile robots, modeled as unknown Markov Decision Processes (MDPs), that are tasked with temporal logic missions, such as sequencing, coverage, or surveillance. The MDP captures uncertainty in the workspace structure and the outcomes of control decisions. The control objective is to synthesize a control policy that maximizes the probability of accomplishing a high-level task, specified as a Linear Temporal Logic (LTL) formula. To address this problem, we propose a novel accelerated model-based reinforcement learning (RL) algorithm for LTL control objectives that is capable of learning control policies significantly faster than related approaches. Its sample-efficiency relies on biasing exploration towards directions that may contribute to task satisfaction. This is accomplished by leveraging an automaton representation of the LTL task as well as a continuously learned MDP model. Finally, we provide comparative experiments that demonstrate the sample efficiency of the proposed method against recent RL methods for LTL objectives.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 18 of its 36 references that are in the map. First posted on arXiv 9 May 2022.

Builds on

18 of 36 references.

References · 18

  1. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 183
  2. 2019Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guaranteescited by 77
  3. 2018Omega-Regular Objectives in Model-Free Reinforcement Learningcited by 74
  4. 2019Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learningcited by 74
  5. 2017Boltzmann Exploration Done Rightcited by 69
  6. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
  7. 2021Compositional Reinforcement Learning from Logical Specificationscited by 65
  8. 2018Logically-Constrained Reinforcement Learningcited by 50
  9. 2014A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specificationscited by 49
  10. 2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logiccited by 38
  11. 2019Reinforcement Learning with Probabilistic Guarantees for Autonomous Drivingcited by 27
  12. 2018STyLuS*: A Temporal Logic Optimal Control Synthesis Algorithm for Large-Scale Multi-Robot Systemscited by 24
6 more
  1. 2020Reinforcement Learning Based Temporal Logic Control with Maximum Probabilistic Satisfactioncited by 22
  2. 2018Logically-Constrained Neural Fitted Q-Iterationcited by 17
  3. 2021Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learningcited by 17
  4. 2020Formal Controller Synthesis for Continuous-Space MDPs via Model-Free Reinforcement Learningcited by 15
  5. 2020Perception-Based Temporal Logic Planning in Uncertain Semantic Mapscited by 12
  6. 2020Continuous Motion Planning with Temporal Logic Specifications using Deep Neural Networkscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 2; 2025: 1; 2026: 1.

9 papers.

  1. 2022Exploiting Transformer in Sparse Reward Reinforcement Learning for Interpretable Temporal Logic Motion Planningcited by 10
  2. 2024Directed Exploration in Reinforcement Learning from Linear Temporal Logiccited by 7
  3. 2023Multi-Agent Reinforcement Learning Guided by Signal Temporal Logic Specificationscited by 6
  4. 2022Learning Minimally-Violating Continuous Control for Infeasible Linear Temporal Logic Specificationscited by 5
  5. 2023Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specificationscited by 3
  6. 2024Sample-Efficient Reinforcement Learning with Temporal Logic Objectives: Leveraging the Task Specification to Guide Explorationcited by 3
  7. 2025Accelerated Learning with Linear Temporal Logic using Differentiable Simulationcited by 1
  8. 2023Verified Compositional Neuro-Symbolic Control for Stochastic Systems with Temporal Logic Taskscited by 0
  9. 2026Live LTL Progress Tracking: Towards Task-Based Explorationcited by 0