mlsanityData to 14 Oct 2026

Q-Learning for Robust Satisfaction of Signal Temporal Logic Specifications

arXiv 1609.07409 ↗First posted 23 Sep 2016eess.SY cs.SYcited by 67

Not on the map. Why?

Cited by 22 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper addresses the problem of learning optimal policies for satisfying signal temporal logic (STL) specifications by agents with unknown stochastic dynamics. The system is modeled as a Markov decision process, in which the states represent partitions of a continuous space and the transition probabilities are unknown. We formulate two synthesis problems where the desired STL specification is enforced by maximizing the probability of satisfaction, and the expected robustness degree, that is, a measure quantifying the quality of satisfaction. We discuss that Q-learning is not directly applicable to these problems because, based on the quantitative semantics of STL, the probability of satisfaction and expected robustness degree are not in the standard objective form of Q-learning. To resolve this issue, we propose an approximation of STL synthesis problems that can be solved via Q-learning, and we derive some performance bounds for the policies obtained by the approximate approach. The performance of the proposed method is demonstrated via simulations.

Key facts. Cited by 67 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Sep 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 13; 2022: 11; 2023: 6; 2024: 9; 2025: 12; 2026: 16.

67 papers.

Sort:

By area:

  1. 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 74
  2. 2021A Survey on Interpretable Reinforcement Learningcited by 73
  3. 2021Compositional Reinforcement Learning from Logical Specificationscited by 67
  4. 2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logiccited by 39
  5. 2022Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarkingcited by 34
  6. 2021DeepSTL -- From English Requirements to Signal Temporal Logiccited by 22
  7. 2024Compositional Automata Embeddings for Goal-Conditioned Reinforcement Learningcited by 19
  8. 2023Sample Efficient Model-free Reinforcement Learning from LTL Specifications with Optimality Guaranteescited by 18
  9. 2021Learning from Demonstrations using Signal Temporal Logiccited by 17
  10. 2023Policy Synthesis and Reinforcement Learning for Discounted LTLcited by 14
  11. 2023Signal Temporal Logic Neural Predictive Controlcited by 13
  12. 2022Joint Learning of Reward Machines and Policies in Environments with Partially Known Semanticscited by 12
38 more
  1. 2024LTL-Constrained Policy Optimization with Cycle Experience Replaycited by 11
  2. 2024Reward Machines for Deep RL in Noisy and Uncertain Environmentscited by 11
  3. 2021Model-Free Reinforcement Learning for Optimal Control of MarkovDecision Processes Under Signal Temporal Logic Specificationscited by 10
  4. 2022Noisy Symbolic Abstractions for Deep RL: A case study with Reward Machinescited by 10
  5. 2022Deep reinforcement learning under signal temporal logic constraints using Lagrangian relaxationcited by 8
  6. 2022Specification-Guided Learning of Nash Equilibria with High Social Welfarecited by 8
  7. 2022Funnel-based Reward Shaping for Signal Temporal Logic Tasks in Reinforcement Learningcited by 8
  8. 2022Model-Free Reinforcement Learning for Symbolic Automata-encoded Objectivescited by 6
  9. 2023Learning Robust and Correct Controllers from Signal Temporal Logic Specifications Using BarrierNetcited by 6
  10. 2025Zero-Shot Trajectory Planning for Signal Temporal Logic Taskscited by 6
  11. 2025TGPO: Temporal Grounded Policy Optimization for Signal Temporal Logic Taskscited by 6
  12. 2023Reinforcement Learning with Knowledge Representation and Reasoning: A Brief Surveycited by 5
  13. 2025Ground-Compose-Reinforce: Grounding Language in Agentic Behaviours using Limited Datacited by 5
  14. 2024Logically Constrained Robotics Transformers for Enhanced Perception-Action Planningcited by 4
  15. 2021Safe Model-based Control from Signal Temporal Logic Specifications Using Recurrent Neural Networkscited by 3
  16. 2021Reinforcement Learning with Formal Performance Metrics for Quadcopter Attitude Control under Non-nominal Contextscited by 3
  17. 2022Learning Performance Graphs from Demonstrations via Task-Based Evaluationscited by 3
  18. 2024Interpretable Imitation Learning via Generative Adversarial STL Inference and Controlcited by 3
  19. 2024Inductive Generalization in Reinforcement Learning from Specificationscited by 3
  20. 2025Pretrained Embeddings as a Behavior Specification Mechanismcited by 3
  21. 2021Reinforcement Learning Based Temporal Logic Control with Soft Constraints Using Limit-deterministic Generalized Buchi Automatacited by 2
  22. 2023Reinforcement Learning with Temporal-Logic-Based Causal Diagramscited by 2
  23. 2025HypRL: Reinforcement Learning of Control Policies for Hyperpropertiescited by 2
  24. 2025Automaton Constrained Q-Learningcited by 2
  25. 2026Learning Gait-Aware Quadruped Locomotion with Temporal Logic Specificationscited by 2
  26. 2021Learning-'N-Flying: A Learning-based, Decentralized Mission Aware UAS Collision Avoidance Schemecited by 1
  27. 2021Deep Reinforcement Learning Based Networked Control with Network Delays for Signal Temporal Logic Specificationscited by 1
  28. 2021Learning Provably Robust Motion Planners Using Funnel Librariescited by 1
  29. 2022Risk-Awareness in Learning Neural Controllers for Temporal Logic Objectivescited by 1
  30. 2022Neural Controller Synthesis for Signal Temporal Logic Specifications Using Encoder-Decoder Structured Networkscited by 1
  31. 2024Multi-agent transformer-accelerated RL for satisfaction of STL specificationscited by 1
  32. 2024Probabilistic Satisfaction of Temporal Logic Constraints in Reinforcement Learning via Adaptive Policy-Switchingcited by 1
  33. 2024Accelerating Proximal Policy Optimization Learning Using Task Prediction for Solving Environments with Delayed Rewardscited by 1
  34. 2025Accelerated Learning with Linear Temporal Logic using Differentiable Simulationcited by 1
  35. 2026Shielded Reinforcement Learning Under Dynamic Temporal Logic Constraintscited by 1
  36. 2026SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learningcited by 1
  37. 2026Runtime Monitoring of Perception-Based Autonomous Systems via Embedding Temporal Logiccited by 1
  38. 2022CatlNet: Learning Communication and Coordination Policies from CaTL+ Specificationscited by 0

50 most cited of 67 All 67 (JSON)