mlsanityData to 7 Oct 2026

Deep reinforcement learning under signal temporal logic constraints using Lagrangian relaxation

arXiv 2201.08504 ↗First posted 21 Jan 2022stat.ML cs.LG cs.SYcited by 8

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (DRL) has attracted much attention as an approach to solve optimal control problems without mathematical models of systems. On the other hand, in general, constraints may be imposed on optimal control problems. In this study, we consider the optimal control problems with constraints to complete temporal control tasks. We describe the constraints using signal temporal logic (STL), which is useful for time sensitive control tasks since it can specify continuous signals within bounded time intervals. To deal with the STL constraints, we introduce an extended constrained Markov decision process (CMDP), which is called a τ-CMDP. We formulate the STL-constrained optimal control problem as the τ-CMDP and propose a two-phase constrained DRL algorithm using the Lagrangian relaxation method. Through simulations, we also demonstrate the learning performance of the proposed algorithm.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 19 of its 35 references that are in the map. First posted on arXiv 21 Jan 2022.

Builds on

19 of 35 references.

References · 19

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  6. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  7. 2018Applications of Deep Reinforcement Learning in Communications and Networking: A Surveycited by 163
  8. 2018Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applicationscited by 122
  9. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 103
  10. 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 72
  11. 2016Q-Learning for Robust Satisfaction of Signal Temporal Logic Specificationscited by 65
  12. 2018Logically-Constrained Reinforcement Learningcited by 50
7 more
  1. 2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logiccited by 38
  2. 2019Modular Deep Reinforcement Learning with Temporal Logic Specificationscited by 25
  3. 2020Tractable Reinforcement Learning of Signal Temporal Logic Objectivescited by 21
  4. 2021Learning from Demonstrations using Signal Temporal Logiccited by 17
  5. 2021Model-Free Reinforcement Learning for Optimal Control of MarkovDecision Processes Under Signal Temporal Logic Specificationscited by 10
  6. 2017Model Predictive Control for Signal Temporal Logic Specificationcited by 8
  7. 2021Deep Reinforcement Learning Based Networked Control with Network Delays for Signal Temporal Logic Specificationscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 0; 2024: 2; 2025: 2; 2026: 2.

8 papers.

  1. 2024LTL-Constrained Policy Optimization with Cycle Experience Replaycited by 10
  2. 2025Zero-Shot Trajectory Planning for Signal Temporal Logic Taskscited by 6
  3. 2025TGPO: Temporal Grounded Policy Optimization for Signal Temporal Logic Taskscited by 5
  4. 2022Funnel-based Reward Shaping for Signal Temporal Logic Tasks in Reinforcement Learningcited by 4
  5. 2022Neural Controller Synthesis for Signal Temporal Logic Specifications Using Encoder-Decoder Structured Networkscited by 1
  6. 2024Concurrent Learning of Policy and Unknown Safety Constraints in Reinforcement Learningcited by 1
  7. 2026Specification-Aware Distribution Shaping for Robotics Foundation Modelscited by 0
  8. 2026Reward Machines for Signal Temporal Logiccited by 0