mlsanityData to 7 Oct 2026

Modular Deep Reinforcement Learning with Temporal Logic Specifications

arXiv 1909.11591 ↗First posted 23 Sep 2019cs.LG cs.AI cs.LOcited by 25

Not on the map. Why?

Abstract (arXiv)

We propose an actor-critic, model-free, and online Reinforcement Learning (RL) framework for continuous-state continuous-action Markov Decision Processes (MDPs) when the reward is highly sparse but encompasses a high-level temporal structure. We represent this temporal structure by a finite-state machine and construct an on-the-fly synchronised product with the MDP and the finite machine. The temporal structure acts as a guide for the RL agent within the product, where a modular Deep Deterministic Policy Gradient (DDPG) architecture is proposed to generate a low-level control policy. We evaluate our framework in a Mars rover experiment and we present the success rate of the synthesised policy.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 7; 2022: 4; 2023: 3; 2024: 2; 2025: 1; 2026: 7.

25 papers.

Sort:

By area:

  1. 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 72
  2. 2021Compositional Reinforcement Learning from Logical Specificationscited by 65
  3. 2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logiccited by 38
  4. 2021The Logical Options Frameworkcited by 24
  5. 2020Active Task-Inference-Guided Deep Inverse Reinforcement Learningcited by 17
  6. 2023Policy Synthesis and Reinforcement Learning for Discounted LTLcited by 14
  7. 2024Reward Machines for Deep RL in Noisy and Uncertain Environmentscited by 11
  8. 2022Noisy Symbolic Abstractions for Deep RL: A case study with Reward Machinescited by 10
  9. 2022Deep reinforcement learning under signal temporal logic constraints using Lagrangian relaxationcited by 8
  10. 2022Specification-Guided Learning of Nash Equilibria with High Social Welfarecited by 8
  11. 2021Lifelong Reinforcement Learning with Temporal Logic Formulas and Reward Machinescited by 5
  12. 2023Reinforcement Learning with Knowledge Representation and Reasoning: A Brief Surveycited by 5
13 more
  1. 2022Possibility Before Utility: Learning And Using Hierarchical Affordancescited by 4
  2. 2024Inductive Generalization in Reinforcement Learning from Specificationscited by 3
  3. 2025HypRL: Reinforcement Learning of Control Policies for Hyperpropertiescited by 2
  4. 2026PlatoLTL: Scaling LTL-Guided Multi-Task RLcited by 2
  5. 2026Learning Gait-Aware Quadruped Locomotion with Temporal Logic Specificationscited by 2
  6. 2021Safer Reinforcement Learning through Transferable Instinct Networkscited by 1
  7. 2021WFA-IRL: Inverse Reinforcement Learning of Autonomous Behaviors Encoded as Weighted Finite Automatacited by 0
  8. 2023Topology-Guided Modular Actor-Critic Learning for Continuous Systems under Temporal Objectivescited by 0
  9. 2026Live LTL Progress Tracking: Towards Task-Based Explorationcited by 0
  10. 2026Reinforcement Learning for Reachability: Guaranteeing Asymptotic Optimalitycited by 0
  11. 2026Decoupled Behavioral Cloning for Scalable Inductive Generalization in RL from Specificationscited by 0
  12. 2026HyPOLE: Hyperproperty-Guided Multi-Agent Reinforcement Learning under Partial Observationcited by 0
  13. 2026Efficient Bayes-Adaptive Reinforcement Learning with Temporal Logic Specificationscited by 0