mlsanityData to 14 Oct 2026

Policy Synthesis and Reinforcement Learning for Discounted LTL

arXiv 2305.17115 ↗First posted 26 May 2023cs.LO cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

The difficulty of manually specifying reward functions has led to an interest in using linear temporal logic (LTL) to express objectives for reinforcement learning (RL). However, LTL has the downside that it is sensitive to small perturbations in the transition probabilities, which prevents probably approximately correct (PAC) learning without additional assumptions. Time discounting provides a way of removing this sensitivity, while retaining the high expressivity of the logic. We study the use of discounted LTL for policy synthesis in Markov decision processes with unknown transition probabilities, and show how to reduce discounted LTL to discounted-sum reward via a reward machine when all discount factors are identical.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 20 of its 45 references that are in the map. First posted on arXiv 26 May 2023.

Builds on

20 of 45 references.

References · 20

  1. 2016Concrete Problems in AI Safetycited by 1,642
  2. 2016Reinforcement Learning With Temporal Logic Rewardscited by 117
  3. 2019Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guaranteescited by 77
  4. 2018Omega-Regular Objectives in Model-Free Reinforcement Learningcited by 75
  5. 2019Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learningcited by 75
  6. 2018Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specificationscited by 72
  7. 2020A Composable Specification Language for Reinforcement Learning Taskscited by 69
  8. 2016Q-Learning for Robust Satisfaction of Signal Temporal Logic Specificationscited by 67
  9. 2017Environment-Independent Task Specifications via GLTLcited by 67
  10. 2021Compositional Reinforcement Learning from Logical Specificationscited by 67
  11. 2014Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraintscited by 54
  12. 2018Logically-Constrained Reinforcement Learningcited by 51
8 more
  1. 2014A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specificationscited by 49
  2. 2019PAC Statistical Model Checking for Markov Decision Processes and Stochastic Gamescited by 32
  3. 2019Transfer of Temporal Logic Formulas in Reinforcement Learningcited by 31
  4. 2019Modular Deep Reinforcement Learning with Temporal Logic Specificationscited by 25
  5. 2015Faster Statistical Model Checking for Unbounded Temporal Propertiescited by 9
  6. 2022Specification-Guided Learning of Nash Equilibria with High Social Welfarecited by 8
  7. 2014Discounting in LTLcited by 6
  8. 2020LTLf Synthesis on Probabilistic Systemscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 4; 2026: 4.

14 papers.

Sort:

By area:

  1. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 21
  2. 2024Compositional Automata Embeddings for Goal-Conditioned Reinforcement Learningcited by 19
  3. 2024Dynamic Model Predictive Shielding for Provably Safe Reinforcement Learningcited by 15
  4. 2023A PAC Learning Algorithm for LTL and Omega-regular Objectives in MDPscited by 13
  5. 2024LTL-Constrained Policy Optimization with Cycle Experience Replaycited by 11
  6. 2025Provably Correct Automata Embeddings for Optimal Automata-Conditioned Reinforcement Learningcited by 6
  7. 2025Pretrained Embeddings as a Behavior Specification Mechanismcited by 3
  8. 2024Integrating Explanations in Learning LTL Specifications from Demonstrationscited by 1
  9. 2026Runtime Monitoring of Perception-Based Autonomous Systems via Embedding Temporal Logiccited by 1
  10. 2025Hyperproperty-Constrained Secure Reinforcement Learningcited by 0
  11. 2025Physics-Informed Reward Machinescited by 0
  12. 2026COOL-MC: Verifying and Explaining RL Policies for Platelet Inventory Managementcited by 0
2 more
  1. 2026COOL-MC: Verifying and Explaining RL Policies for Multi-bridge Network Maintenancecited by 0
  2. 2026Live LTL Progress Tracking: Towards Task-Based Explorationcited by 0