mlsanityData to 13 Oct 2026

Policy Optimization with Linear Temporal Logic Constraints

arXiv 2206.09546 ↗First posted 20 Jun 2022cs.LG cs.AI cs.LOcited by 19

Not on the map. Why?

Abstract (arXiv)

We study the problem of policy optimization (PO) with linear temporal logic (LTL) constraints. The language of LTL allows flexible description of tasks that may be unnatural to encode as a scalar cost function. We consider LTL-constrained PO as a systematic framework, decoupling task specification from policy selection, and as an alternative to the standard of cost shaping. With access to a generative model, we develop a model-based approach that enjoys a sample complexity analysis for guaranteeing both task satisfaction and cost optimality (through a reduction to a reachability problem). Empirically, our algorithm can achieve strong performance even in low-sample regimes.

Key facts. Cited by 19 papers in the map (data through 13 Oct 2026). Builds on 31 of its 71 references that are in the map. First posted on arXiv 20 Jun 2022.

Builds on

31 of 71 references.

References · 31

  1. 2017CARLA: An Open Urban Driving Simulatorcited by 2,439
  2. 2016OpenAI Gymcited by 1,694
  3. 2017Constrained Policy Optimizationcited by 931
  4. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  5. 2019Batch Policy Learning under Constraintscited by 243
  6. 2013(More) Efficient Reinforcement Learning via Posterior Samplingcited by 233
  7. 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 233
  8. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 183
  9. 2019On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shiftcited by 174
  10. 2019Model-Based Reinforcement Learning with a Generative Model is Minimax Optimalcited by 126
  11. 2020Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Modelcited by 109
  12. 2015Sample Complexity of Episodic Fixed-Horizon Reinforcement Learningcited by 105
19 more
  1. 2020Finite-Time Analysis of Asynchronous Stochastic Approximation and Q-Learningcited by 94
  2. 2018Hierarchical Imitation and Reinforcement Learningcited by 88
  3. 2019Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learningcited by 74
  4. 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 72
  5. 2021On the Expressivity of Markov Rewardcited by 71
  6. 2020A Composable Specification Language for Reinforcement Learning Taskscited by 68
  7. 2017Environment-Independent Task Specifications via GLTLcited by 67
  8. 2021On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learningcited by 59
  9. 2016Conservative Contextual Linear Banditscited by 57
  10. 2019Reinforcement Learning with Convex Constraintscited by 57
  11. 2014Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraintscited by 53
  12. 2016Reinforcement Learning of POMDPs using Spectral Methodscited by 51
  13. 2018Logically-Constrained Reinforcement Learningcited by 50
  14. 2014A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specificationscited by 49
  15. 2014Verification of Markov Decision Processes using Learning Algorithmscited by 38
  16. 2021Stochastic Shortest Path: Minimax, Parameter-Free and Towards Horizon-Free Regretcited by 34
  17. 2019PAC Statistical Model Checking for Markov Decision Processes and Stochastic Gamescited by 30
  18. 2020Improved Analysis of UCRL2 with Empirical Bernstein Inequalitycited by 18
  19. 2020Optimal Probabilistic Motion Planning with Potential Infeasible LTL Constraintscited by 14

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 4; 2024: 9; 2025: 2; 2026: 4.

19 papers.

Sort:

By area:

  1. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 20
  2. 2023Eventual Discounting Temporal Logic Counterfactual Experience Replaycited by 18
  3. 2024Compositional Automata Embeddings for Goal-Conditioned Reinforcement Learningcited by 18
  4. 2024LTLDoG: Satisfying Temporally-Extended Symbolic Constraints for Safe Diffusion-based Planningcited by 16
  5. 2024Reinforcement Learning with LTL and ω-Regular Objectives via Optimality-Preserving Translation to Average Rewardscited by 13
  6. 2023A PAC Learning Algorithm for LTL and Omega-regular Objectives in MDPscited by 12
  7. 2024LTL-Constrained Policy Optimization with Cycle Experience Replaycited by 10
  8. 2024Directed Exploration in Reinforcement Learning from Linear Temporal Logiccited by 7
  9. 2024Diffusion Meets Options: Hierarchical Generative Skill Composition for Temporally-Extended Taskscited by 7
  10. 2023Reinforcement Learning with Knowledge Representation and Reasoning: A Brief Surveycited by 5
  11. 2024Regret-Free Reinforcement Learning for LTL Specificationscited by 1
  12. 2025Accelerated Learning with Linear Temporal Logic using Differentiable Simulationcited by 1
7 more
  1. 2026Safe Reinforcement Learning via Recovery-based Shielding with Gaussian Process Dynamics Modelscited by 1
  2. 2023Using Experience Classification for Training Non-Markovian Taskscited by 0
  3. 2024Belief-State Query Policies for User-Aligned POMDPscited by 0
  4. 2025Near-Optimal Reinforcement Learning for Constrained Recurrence Objectivescited by 0
  5. 2026Learning Bilevel Policies over Symbolic World Models for Long-Horizon Planningcited by 0
  6. 2026Efficient Bayes-Adaptive Reinforcement Learning with Temporal Logic Specificationscited by 0
  7. 2026Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RLcited by 0