mlsanityData to 14 Oct 2026

Zero-Shot Instruction Following in RL via Structured LTL Representations

arXiv 2512.02633 ↗First posted 2 Dec 2025cs.AI cs.LGcited by 0

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

Linear temporal logic (LTL) is a compelling framework for specifying complex, structured tasks for reinforcement learning (RL) agents. Recent work has shown that interpreting LTL instructions as finite automata, which can be seen as high-level programs monitoring task progress, enables learning a single generalist policy capable of executing arbitrary instructions at test time. However, existing approaches fall short in environments where multiple high-level events (i.e., atomic propositions) can be true at the same time and potentially interact in complicated ways. In this work, we propose a novel approach to learning a multi-task policy for following arbitrary LTL instructions that addresses this shortcoming. Our method conditions the policy on sequences of simple Boolean formulae, which directly align with transitions in the automaton, and are encoded via a graph neural network (GNN) to yield structured task representations. Experiments in a complex chess-based environment demonstrate the advantages of our approach.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 40 of its 58 references that are in the map. First posted on arXiv 2 Dec 2025.

Builds on

40 of 58 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learning3 Aug
  2. 2024Compositional Automata Embeddings for Goal-Conditioned Reinforcement Learning31 Oct
  3. 2024Reinforcement Learning with LTL and ω-Regular Objectives via Optimality-Preserving Translation to Average Rewards16 Oct
  4. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RL6 Oct
  5. 2024LTL-Constrained Policy Optimization with Cycle Experience Replay17 Apr

Other references · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2016Layer Normalizationcited by 5,669
  5. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
  6. 2017Deep Setscited by 1,421
  7. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
  8. 2017Hindsight Experience Replaycited by 1,136
  9. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  10. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  11. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 195
  12. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
23 more
  1. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  2. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 146
  3. 2019Using Natural Language for Reward Shaping in Reinforcement Learningcited by 114
  4. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 98
  5. 2018Omega-Regular Objectives in Model-Free Reinforcement Learningcited by 75
  6. 2019Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learningcited by 75
  7. 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 74
  8. 2021Compositional Reinforcement Learning from Logical Specificationscited by 67
  9. 2020Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Textcited by 59
  10. 2022Reachability Constrained Reinforcement Learningcited by 59
  11. 2014Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraintscited by 54
  12. 2018Logically-Constrained Reinforcement Learningcited by 51
  13. 2014A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specificationscited by 49
  14. 2020Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulascited by 36
  15. 2021The Logical Options Frameworkcited by 25
  16. 2018Can Neural Networks Understand Logical Entailment?cited by 24
  17. 2024MaestroMotif: Skill Design from Artificial Intelligence Feedbackcited by 24
  18. 2020Reinforcement Learning Based Temporal Logic Control with Maximum Probabilistic Satisfactioncited by 22
  19. 2020Systematic Generalisation through Task Temporal Logic and Deep Reinforcement Learningcited by 21
  20. 2019Improving Graph Neural Network Representations of Logical Formulae with Subgraph Poolingcited by 20
  21. 2023Eventual Discounting Temporal Logic Counterfactual Experience Replaycited by 19
  22. 2023Sample Efficient Model-free Reinforcement Learning from LTL Specifications with Optimality Guaranteescited by 18
  23. 2021In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specificationscited by 17

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Zero-Shot Instruction Following in RL via Structured LTL Representations
  2. 2026Semantically Labelled Automata for Multi-Task Reinforcement Learning with LTL Instructions
  3. 2026PlatoLTL: Scaling LTL-Guided Multi-Task RL
  4. 2026Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RL
  5. 2024Comp-LTL: Temporal Logic Planning via Zero-Shot Policy Composition
  6. 2026Grounding LTL Tasks in Sub-Symbolic RL Environments for Zero-Shot Generalization
  7. 2026Temporal Logic Guided Universal Task Representations for Reinforcement Learning
  8. 2025Accelerated Learning with Linear Temporal Logic using Differentiable Simulation