Zero-Shot Instruction Following in RL via Structured LTL Representations
Not on the map. Why?
Abstract (arXiv)
We study instruction following in multi-task reinforcement learning, where an agent must zero-shot execute novel tasks not seen during training. In this setting, linear temporal logic (LTL) has recently been adopted as a powerful framework for specifying structured, temporally extended tasks. While existing approaches successfully train generalist policies, they often struggle to effectively capture the rich logical and temporal structure inherent in LTL specifications. In this work, we address these concerns with a novel approach to learn structured task representations that facilitate training and generalisation. Our method conditions the policy on sequences of Boolean formulae constructed from a finite automaton of the task. We propose a hierarchical neural architecture to encode the logical structure of these formulae, and introduce an attention mechanism that enables the policy to reason about future subgoals. Experiments in a variety of complex environments demonstrate the strong generalisation capabilities and superior performance of our approach.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 41 of its 58 references that are in the map. First posted on arXiv 15 Feb 2026.
Builds on
41 of 58 references.
References · 41
- 2017Attention Is All You Needcited by 59,932
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2016Layer Normalizationcited by 5,579
- 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
- 2017Deep Setscited by 1,394
- 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
- 2017Hindsight Experience Replaycited by 1,096
- 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
- 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
- 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 193
- 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
29 more
- 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
- 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
- 2019Using Natural Language for Reward Shaping in Reinforcement Learningcited by 114
- 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 97
- 2018Omega-Regular Objectives in Model-Free Reinforcement Learningcited by 74
- 2019Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learningcited by 74
- 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 72
- 2021Compositional Reinforcement Learning from Logical Specificationscited by 65
- 2020Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Textcited by 59
- 2014Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraintscited by 53
- 2022Reachability Constrained Reinforcement Learningcited by 52
- 2018Logically-Constrained Reinforcement Learningcited by 50
- 2014A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specificationscited by 49
- 2020Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulascited by 36
- 2018Can Neural Networks Understand Logical Entailment?cited by 24
- 2021The Logical Options Frameworkcited by 24
- 2024MaestroMotif: Skill Design from Artificial Intelligence Feedbackcited by 24
- 2020Reinforcement Learning Based Temporal Logic Control with Maximum Probabilistic Satisfactioncited by 22
- 2020Systematic Generalisation through Task Temporal Logic and Deep Reinforcement Learningcited by 21
- 2019Improving Graph Neural Network Representations of Logical Formulae with Subgraph Poolingcited by 20
- 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 20
- 2023Eventual Discounting Temporal Logic Counterfactual Experience Replaycited by 18
- 2024Compositional Automata Embeddings for Goal-Conditioned Reinforcement Learningcited by 18
- 2021In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specificationscited by 17
- 2023Sample Efficient Model-free Reinforcement Learning from LTL Specifications with Optimality Guaranteescited by 17
- 2024Reinforcement Learning with LTL and ω-Regular Objectives via Optimality-Preserving Translation to Average Rewardscited by 13
- 2024LTL-Constrained Policy Optimization with Cycle Experience Replaycited by 10
- 2025One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learningcited by 8
- 2026PlatoLTL: Scaling LTL-Guided Multi-Task RLcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Zero-Shot Instruction Following in RL via Structured LTL Representations
- 2026hint2: Hierarchical World Models for Inference-Time Temporal Logic Guidance
- 2022Temporal Logic Imitation: Learning Plan-Satisficing Motion Policies from Demonstrations
- 2026Breaking the Capability Ceiling of LLM Post-Training by Reintroducing Markov States
- 2024Foundation Policies with Hilbert Representations
- 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild
- 2026Neuro-Symbolic Injection of LTLf Constraints in Autoregressive Reinforcement Learning Policies
- 2026Reward-Zero: Language Embedding Driven Implicit Reward Mechanisms for Reinforcement Learning