mlsanityData to 14 Oct 2026

One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learning

arXiv 2508.01561 ↗First posted 3 Aug 2025cs.AIcited by 8

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

Generalizing to complex and temporally extended task objectives and safety constraints remains a critical challenge in reinforcement learning (RL). Linear temporal logic (LTL) offers a unified formalism to specify such requirements, yet existing methods are limited in their abilities to handle nested long-horizon tasks and safety constraints, and cannot identify situations when a subgoal is not satisfiable and an alternative should be sought. In this paper, we introduce GenZ-LTL, a method that enables zero-shot generalization to arbitrary LTL specifications. GenZ-LTL leverages the structure of B"uchi automata to decompose an LTL task specification into sequences of reach-avoid subgoals. Contrary to the current state-of-the-art method that conditions on subgoal sequences, we show that it is more effective to achieve zero-shot generalization by solving these reach-avoid problems textitone subgoal at a time through proper safe RL formulations. In addition, we introduce a novel subgoal-induced observation reduction technique that can mitigate the exponential complexity of subgoal-state combinations under realistic assumptions. Empirical results show that GenZ-LTL substantially outperforms existing methods in zero-shot generalization to unseen LTL specifications.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 47 of its 61 references that are in the map. First posted on arXiv 3 Aug 2025.

Builds on

47 of 61 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Provably Correct Automata Embeddings for Optimal Automata-Conditioned Reinforcement Learning6 Mar
  2. 2024Compositional Automata Embeddings for Goal-Conditioned Reinforcement Learning31 Oct
  3. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RL6 Oct
  4. 2024Directed Exploration in Reinforcement Learning from Linear Temporal Logic18 Aug
  5. 2024LTL-Constrained Policy Optimization with Cycle Experience Replay17 Apr

Other references · 42

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  3. 2017Constrained Policy Optimizationcited by 983
  4. 2018Safe Exploration in Continuous Action Spacescited by 247
  5. 2020Projection-Based Constrained Policy Optimizationcited by 203
  6. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  7. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
  8. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
  9. 2016Reinforcement Learning With Temporal Logic Rewardscited by 117
  10. 2020Conservative Safety Critics for Explorationcited by 113
  11. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 98
  12. 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 74
30 more
  1. 2020A Composable Specification Language for Reinforcement Learning Taskscited by 69
  2. 2024Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Modelcited by 64
  3. 2022Reachability Constrained Reinforcement Learningcited by 59
  4. 2014Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraintscited by 54
  5. 2018Logically-Constrained Reinforcement Learningcited by 51
  6. 2023Multi-Task Reinforcement Learning with Mixture of Orthogonal Expertscited by 49
  7. 2020A Boolean Task Algebra for Reinforcement Learningcited by 48
  8. 2021Learning Markov State Abstractions for Deep Reinforcement Learningcited by 44
  9. 2022Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learningcited by 38
  10. 2023Iterative Reachability Estimation for Safe Reinforcement Learningcited by 37
  11. 2022Model-based Safe Deep Reinforcement Learning via a Constrained Proximal Policy Optimization Algorithmcited by 29
  12. 2023Safe Offline Reinforcement Learning with Real-Time Budget Constraintscited by 28
  13. 2023Solving Stabilize-Avoid Optimal Control via Epigraph Form and Deep Reinforcement Learningcited by 24
  14. 2020Systematic Generalisation through Task Temporal Logic and Deep Reinforcement Learningcited by 21
  15. 2023Trust Region-Based Safe Distributional Reinforcement Learning for Multiple Constraintscited by 21
  16. 2023Eventual Discounting Temporal Logic Counterfactual Experience Replaycited by 19
  17. 2023Sample Efficient Model-free Reinforcement Learning from LTL Specifications with Optimality Guaranteescited by 18
  18. 2021In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specificationscited by 17
  19. 2024Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement Learningcited by 17
  20. 2024Hamilton-Jacobi Reachability in Reinforcement Learning: A Surveycited by 16
  21. 2022Probabilistic Safeguard for Reinforcement Learning Using Safety Index Guided Gaussian Process Modelscited by 15
  22. 2023Learning Shared Safety Constraints from Multi-task Demonstrationscited by 15
  23. 2022Safe and Efficient Reinforcement Learning Using Disturbance-Observer-Based Control Barrier Functionscited by 11
  24. 2023Provably Safe Reinforcement Learning with Step-wise Violation Constraintscited by 11
  25. 2023Understanding and Addressing the Pitfalls of Bisimulation-based Representations in Offline Reinforcement Learningcited by 10
  26. 2024Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learningcited by 10
  27. 2023Meta-learning in healthcare: A surveycited by 9
  28. 2023Constrained Meta-Reinforcement Learning for Adaptable Safety Guarantee with Differentiable Convex Programmingcited by 8
  29. 2024Efficient and Generalized end-to-end Autonomous Driving System with Latent Deep Reinforcement Learning and Demonstrationscited by 7
  30. 2024Automated Multi-Task Learning for Joint Disease Prediction on Electronic Health Recordscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2026PlatoLTL: Scaling LTL-Guided Multi-Task RLcited by 2
  2. 2026Semantically Labelled Automata for Multi-Task Reinforcement Learning with LTL Instructionscited by 2
  3. 2025Automata-Conditioned Cooperative Multi-Agent Reinforcement Learningcited by 1
  4. 2026Zero-Shot Instruction Following in RL via Structured LTL Representationscited by 1
  5. 2026SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learningcited by 1
  6. 2025Zero-Shot Instruction Following in RL via Structured LTL Representationscited by 0
  7. 2026Decoupled Behavioral Cloning for Scalable Inductive Generalization in RL from Specificationscited by 0
  8. 2026Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Comp-LTL: Temporal Logic Planning via Zero-Shot Policy Composition
  2. 2024Inductive Generalization in Reinforcement Learning from Specifications
  3. 2026Grounding LTL Tasks in Sub-Symbolic RL Environments for Zero-Shot Generalization
  4. 2025Accelerated Learning with Linear Temporal Logic using Differentiable Simulation
  5. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
  6. 2025A Unified Framework for Zero-Shot Reinforcement Learning
  7. 2025Zero-Shot Reinforcement Learning Under Partial Observability
  8. 2025Automaton Constrained Q-Learning