mlsanityData to 7 Oct 2026

Overcoming Exploration: Deep Reinforcement Learning for Continuous Control in Cluttered Environments from Temporal Logic Specifications

arXiv 2201.12231 ↗First posted 28 Jan 2022cs.RO cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Model-free continuous control for robot navigation tasks using Deep Reinforcement Learning (DRL) that relies on noisy policies for exploration is sensitive to the density of rewards. In practice, robots are usually deployed in cluttered environments, containing many obstacles and narrow passageways. Designing dense effective rewards is challenging, resulting in exploration issues during training. Such a problem becomes even more serious when tasks are described using temporal logic specifications. This work presents a deep policy gradient algorithm for controlling a robot with unknown dynamics operating in a cluttered environment when the task is specified as a Linear Temporal Logic (LTL) formula. To overcome the environmental challenge of exploration during training, we propose a novel path planning-guided reward scheme by integrating sampling-based methods to effectively complete goal-reaching missions. To facilitate LTL satisfaction, our approach decomposes the LTL mission into sub-goal-reaching tasks that are solved in a distributed manner. Our framework is shown to significantly improve performance (effectiveness, efficiency) and exploration of robots tasked with complex missions in large-scale cluttered environments. A video demonstration can be found on YouTube Channel: https://youtu.be/yMh_NUNWxho.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 20 of its 42 references that are in the map. First posted on arXiv 28 Jan 2022.

Builds on

20 of 42 references.

References · 20

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  8. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  9. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 183
  10. 2021Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functionscited by 91
  11. 2017Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learningcited by 81
  12. 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 72
8 more
  1. 2019Joint Inference of Reward Machines and Policies for Reinforcement Learningcited by 56
  2. 2015Control Contraction Metrics: Convex and Intrinsic Criteria for Nonlinear Feedback Designcited by 38
  3. 2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logiccited by 38
  4. 2018STyLuS*: A Temporal Logic Optimal Control Synthesis Algorithm for Large-Scale Multi-Robot Systemscited by 24
  5. 2022Learning Safe, Generalizable Perception-based Hybrid Control with Certificatescited by 24
  6. 2019An Abstraction-Free Method for Multi-Robot Temporal Logic Optimal Control Synthesiscited by 11
  7. 2019Control of Mobile Robots Using Barrier Functions Under Temporal Logic Specificationscited by 8
  8. 2019DeepSynth: Automata Synthesis for Automatic Task Segmentation in Deep Reinforcement Learningcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 4; 2025: 1; 2026: 2.

11 papers.

  1. 2023LgTS: Dynamic Task Sampling using LLM-generated sub-goals for Reinforcement Learning Agentscited by 11
  2. 2022Exploiting Transformer in Sparse Reward Reinforcement Learning for Interpretable Temporal Logic Motion Planningcited by 10
  3. 2022Learning Minimally-Violating Continuous Control for Infeasible Linear Temporal Logic Specificationscited by 5
  4. 2024Adaptive Reward Design for Reinforcement Learningcited by 5
  5. 2024Robust Imitation Learning for Automated Game Testingcited by 4
  6. 2023Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specificationscited by 3
  7. 2024Sample-Efficient Reinforcement Learning with Temporal Logic Objectives: Leveraging the Task Specification to Guide Explorationcited by 3
  8. 2026PlatoLTL: Scaling LTL-Guided Multi-Task RLcited by 2
  9. 2024Accelerating Proximal Policy Optimization Learning Using Task Prediction for Solving Environments with Delayed Rewardscited by 1
  10. 2025Accelerated Learning with Linear Temporal Logic using Differentiable Simulationcited by 1
  11. 2026SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learningcited by 1