mlsanityData to 7 Oct 2026

Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic

arXiv 2102.12855 ↗First posted 24 Feb 2021cs.LG cs.AI cs.FLcited by 38

Not on the map. Why?

Abstract (arXiv)

This paper investigates the motion planning of autonomous dynamical systems modeled by Markov decision processes (MDP) with unknown transition probabilities over continuous state and action spaces. Linear temporal logic (LTL) is used to specify high-level tasks over infinite horizon, which can be converted into a limit deterministic generalized B"uchi automaton (LDGBA) with several accepting sets. The novelty is to design an embedded product MDP (EP-MDP) between the LDGBA and the MDP by incorporating a synchronous tracking-frontier function to record unvisited accepting sets of the automaton, and to facilitate the satisfaction of the accepting conditions. The proposed LDGBA-based reward shaping and discounting schemes for the model-free reinforcement learning (RL) only depend on the EP-MDP states and can overcome the issues of sparse rewards. Rigorous analysis shows that any RL method that optimizes the expected discounted return is guaranteed to find an optimal policy whose traces maximize the satisfaction probability. A modular deep deterministic policy gradient (DDPG) is then developed to generate such policies over continuous state and action spaces. The performance of our framework is evaluated via an array of OpenAI gym environments.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 21 of its 49 references that are in the map. First posted on arXiv 24 Feb 2021.

Builds on

21 of 49 references.

References · 21

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2019Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guaranteescited by 77
  3. 2018Omega-Regular Objectives in Model-Free Reinforcement Learningcited by 74
  4. 2019Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learningcited by 74
  5. 2016Q-Learning for Robust Satisfaction of Signal Temporal Logic Specificationscited by 65
  6. 2014Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraintscited by 53
  7. 2018Logically-Constrained Reinforcement Learningcited by 50
  8. 2014A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specificationscited by 49
  9. 2020Cautious Reinforcement Learning with Logical Constraintscited by 44
  10. 2019Certified Reinforcement Learning with Logic Guidancecited by 35
  11. 2019Modular Deep Reinforcement Learning with Temporal Logic Specificationscited by 25
  12. 2020Reinforcement Learning Based Temporal Logic Control with Maximum Probabilistic Satisfactioncited by 22
9 more
  1. 2020Tractable Reinforcement Learning of Signal Temporal Logic Objectivescited by 21
  2. 2018Logically-Constrained Neural Fitted Q-Iterationcited by 17
  3. 2020Active Task-Inference-Guided Deep Inverse Reinforcement Learningcited by 17
  4. 2020Formal Controller Synthesis for Continuous-Space MDPs via Model-Free Reinforcement Learningcited by 15
  5. 2020Reinforcement Learning of Control Policy for Linear Temporal Logic Specifications Using Limit-Deterministic Generalized B"uchi Automatacited by 11
  6. 2020Learning Non-Markovian Reward Models in MDPscited by 11
  7. 2019DeepSynth: Automata Synthesis for Automatic Task Segmentation in Deep Reinforcement Learningcited by 8
  8. 2020Continuous Motion Planning with Temporal Logic Specifications using Deep Neural Networkscited by 8
  9. 2020Formal Policy Synthesis for Continuous-Space Systems via Reinforcement Learningcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 9; 2023: 8; 2024: 4; 2025: 4; 2026: 6.

38 papers.

Sort:

By area:

  1. 2021Self-Supervised Online Reward Shaping in Sparse-Reward Environmentscited by 34
  2. 2022Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarkingcited by 31
  3. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 20
  4. 2023Eventual Discounting Temporal Logic Counterfactual Experience Replaycited by 18
  5. 2023Sample Efficient Model-free Reinforcement Learning from LTL Specifications with Optimality Guaranteescited by 17
  6. 2022Automaton-Based Representations of Task Knowledge from Generative Language Modelscited by 12
  7. 2022Overcoming Exploration: Deep Reinforcement Learning for Continuous Control in Cluttered Environments from Temporal Logic Specificationscited by 11
  8. 2022LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learningcited by 11
  9. 2022Exploiting Transformer in Sparse Reward Reinforcement Learning for Interpretable Temporal Logic Motion Planningcited by 10
  10. 2022Accelerated Reinforcement Learning for Temporal Logic Control Objectivescited by 9
  11. 2022Deep reinforcement learning under signal temporal logic constraints using Lagrangian relaxationcited by 8
  12. 2024Directed Exploration in Reinforcement Learning from Linear Temporal Logiccited by 7
26 more
  1. 2024Neural Reward Machinescited by 6
  2. 2022Learning Minimally-Violating Continuous Control for Infeasible Linear Temporal Logic Specificationscited by 5
  3. 2025ARM-FM: Automated Reward Machines via Foundation Models for Compositional Reinforcement Learningcited by 5
  4. 2021Formulation and validation of a car-following model based on deep reinforcement learningcited by 4
  5. 2021Multi-agent Soft Actor-Critic Based Hybrid Motion Planner for Mobile Robotscited by 4
  6. 2021Safe Model-based Control from Signal Temporal Logic Specifications Using Recurrent Neural Networkscited by 3
  7. 2023Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specificationscited by 3
  8. 2024On the Uniqueness of Solution for the Bellman Equation of LTL Objectivescited by 3
  9. 2023Gaussian Process on the Product of Directional Manifoldscited by 2
  10. 2023System III: Learning with Domain Knowledge for Safety Constraintscited by 2
  11. 2023Using Implicit Behavior Cloning and Dynamic Movement Primitive to Facilitate Reinforcement Learning for Robot Motion Planningcited by 2
  12. 2026PlatoLTL: Scaling LTL-Guided Multi-Task RLcited by 2
  13. 2021Safety-Critical Learning of Robot Control with Temporal Logic Specificationscited by 1
  14. 2021Intelligent Traffic Light via Policy-based Deep Reinforcement Learningcited by 1
  15. 2023Model-free Motion Planning of Autonomous Agents for Complex Tasks in Partially Observable Environmentscited by 1
  16. 2023Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesiscited by 1
  17. 2025Accelerated Learning with Linear Temporal Logic using Differentiable Simulationcited by 1
  18. 2021WFA-IRL: Inverse Reinforcement Learning of Autonomous Behaviors Encoded as Weighted Finite Automatacited by 0
  19. 2022CatlNet: Learning Communication and Coordination Policies from CaTL+ Specificationscited by 0
  20. 2025Control Synthesis of Cyber-Physical Systems for Real-Time Specifications through Causation-Guided Reinforcement Learningcited by 0
  21. 2025Reinforcement learning with timed constraints for robotics motion planningcited by 0
  22. 2026DeepDFA: Injecting Temporal Logic in Deep Learning for Sequential Subsymbolic Applicationscited by 0
  23. 2026Do It for HER: First-Order Temporal Logic Reward Specification in Reinforcement Learning (Extended Version)cited by 0
  24. 2026NeuroNL2LTL: A Neurosymbolic Framework for Natural Language Translation of Linear Temporal Logiccited by 0
  25. 2026Temporal Logic Guided Universal Task Representations for Reinforcement Learningcited by 0
  26. 2026Efficient Bayes-Adaptive Reinforcement Learning with Temporal Logic Specificationscited by 0