Semantically Labelled Automata for Multi-Task Reinforcement Learning with LTL Instructions
Not on the map. Why?
Abstract (arXiv)
We study multi-task reinforcement learning (RL), a setting in which an agent learns a single, universal policy capable of generalising to arbitrary, possibly unseen tasks. We consider tasks specified as linear temporal logic (LTL) formulae, which are commonly used in formal methods to specify properties of systems, and have recently been successfully adopted in RL. In this setting, we present a novel task embedding technique leveraging a new generation of semantic LTL-to-automata translations, originally developed for temporal synthesis. The resulting semantically labelled automata contain rich, structured information in each state that allow us to (i) compute the automaton efficiently on-the-fly, (ii) extract expressive task embeddings used to condition the policy, and (iii) naturally support full LTL. Experimental results in a variety of domains demonstrate that our approach achieves state-of-the-art performance and is able to scale to complex specifications where existing methods fail.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 26 of its 43 references that are in the map. First posted on arXiv 6 Feb 2026.
Builds on
26 of 43 references.
References · 26
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
- 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
- 2017Deep Setscited by 1,394
- 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
- 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
- 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 97
- 2016Learning values across many orders of magnitudecited by 80
- 2018Omega-Regular Objectives in Model-Free Reinforcement Learningcited by 74
- 2019Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learningcited by 74
- 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 72
14 more
- 2014A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specificationscited by 49
- 2014Verification of Markov Decision Processes using Learning Algorithmscited by 38
- 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 20
- 2023Eventual Discounting Temporal Logic Counterfactual Experience Replaycited by 18
- 2021In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specificationscited by 17
- 2022LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learningcited by 11
- 2025One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learningcited by 8
- 2025SemML: Enhancing Automata-Theoretic LTL Synthesis with Machine Learningcited by 6
- 2019Semantic Labelling and Learning for Parity Game Solving in LTL Synthesiscited by 5
- 2023Guessing Winning Policies in LTL Synthesis by Semantic Learningcited by 4
- 2024LTLf+ and PPLTL+: Extending LTLf and PPLTL to Infinite Tracescited by 4
- 2020An Efficient Normalisation Procedure for Linear Temporal Logic and Very Weak Alternating Automatacited by 3
- 2025Solving MDPs with LTLf+ and PPLTL+ Temporal Objectivescited by 3
- 2025Good-for-MDP State Reduction for Stochastic LTL Planningcited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.