mlsanityData to 14 Oct 2026

LTL2Action: Generalizing LTL Instructions for Multi-Task RL

arXiv 2102.06858 ↗First posted 13 Feb 2021cs.AI cs.LG cs.ROcited by 74

Not on the map. Why?

Cited by 33 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We address the problem of teaching a deep reinforcement learning (RL) agent to follow instructions in multi-task environments. Instructions are expressed in a well-known formal language -- linear temporal logic (LTL) -- and can specify a diversity of complex, temporally extended behaviours, including conditionals and alternative realizations. Our proposed learning approach exploits the compositional syntax and the semantics of LTL, enabling our RL agent to learn task-conditioned policies that generalize to new instructions, not observed during training. To reduce the overhead of learning LTL semantics, we introduce an environment-agnostic LTL pretraining scheme which improves sample-efficiency in downstream environments. Experiments on discrete and continuous domains target combinatorial task sets of up to ~1039 unique tasks and demonstrate the strength of our approach in learning to solve (unseen) tasks, given LTL instructions.

Key facts. Cited by 74 papers in the map (data through 14 Oct 2026). Builds on 33 of its 62 references that are in the map. First posted on arXiv 13 Feb 2021.

Builds on

33 of 62 references.

References · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
  3. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  4. 2017Learning to Represent Programs with Graphscited by 220
  5. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 203
  6. 2018Learning a SAT Solver from Single-Bit Supervisioncited by 200
  7. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 191
  8. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  9. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  10. 2016Reinforcement Learning With Temporal Logic Rewardscited by 117
  11. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  12. 2019Compositional generalization through meta sequence-to-sequence learningcited by 99
21 more
  1. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 98
  2. 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
  3. 2017Neural Task Programming: Learning to Generalize Across Hierarchical Taskscited by 76
  4. 2017Gated-Attention Architectures for Task-Oriented Language Groundingcited by 72
  5. 2020A Composable Specification Language for Reinforcement Learning Taskscited by 69
  6. 2016Q-Learning for Robust Satisfaction of Signal Temporal Logic Specificationscited by 67
  7. 2017Environment-Independent Task Specifications via GLTLcited by 67
  8. 2020Grounded Language Learning Fast and Slowcited by 54
  9. 2018Logically-Constrained Reinforcement Learningcited by 51
  10. 2018Hierarchical Reinforcement Learning for Zero-shot Generalization with Subtask Dependenciescited by 43
  11. 2020Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulascited by 36
  12. 2019Transfer of Temporal Logic Formulas in Reinforcement Learningcited by 31
  13. 2019Modular Deep Reinforcement Learning with Temporal Logic Specificationscited by 25
  14. 2020Learning a natural-language to LTL executable semantic parser for grounded roboticscited by 22
  15. 2018Interactive Grounded Language Acquisition and Generalization in a 2D Worldcited by 21
  16. 2020Systematic Generalisation through Task Temporal Logic and Deep Reinforcement Learningcited by 21
  17. 2017A Policy Search Method For Temporal Logic Specified Reinforcement Learning Taskscited by 20
  18. 2017Programmable Agentscited by 14
  19. 2017Interpretable Apprenticeship Learning with Temporal Logic Specificationscited by 13
  20. 2019Planning With Uncertain Specifications (PUnS)cited by 11
  21. 2020Learning Branching Heuristics for Propositional Model Countingcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 15; 2023: 9; 2024: 15; 2025: 13; 2026: 20.

74 papers.

Sort:

By area:

  1. 2021Compositional Reinforcement Learning from Logical Specificationscited by 67
  2. 2023Creative Agents: Empowering Agents with Imagination for Creative Taskscited by 24
  3. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 21
  4. 2022Policy Optimization with Linear Temporal Logic Constraintscited by 19
  5. 2023Eventual Discounting Temporal Logic Counterfactual Experience Replaycited by 19
  6. 2024Compositional Automata Embeddings for Goal-Conditioned Reinforcement Learningcited by 19
  7. 2023Constraint-Conditioned Policy Optimization for Versatile Safe Reinforcement Learningcited by 18
  8. 2021In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specificationscited by 17
  9. 2025General agents contain world modelscited by 17
  10. 2024LTLDoG: Satisfying Temporally-Extended Symbolic Constraints for Safe Diffusion-based Planningcited by 16
  11. 2023Compositional Policy Learning in Stochastic Control Systems with Formal Guaranteescited by 14
  12. 2024BlendRL: A Framework for Merging Symbolic and Neural Policy Learningcited by 14
38 more
  1. 2022Overcoming Exploration: Deep Reinforcement Learning for Continuous Control in Cluttered Environments from Temporal Logic Specificationscited by 12
  2. 2022Learning to Follow Instructions in Text-Based Gamescited by 12
  3. 2024LTL-Constrained Policy Optimization with Cycle Experience Replaycited by 11
  4. 2024Reward Machines for Deep RL in Noisy and Uncertain Environmentscited by 11
  5. 2022Exploiting Transformer in Sparse Reward Reinforcement Learning for Interpretable Temporal Logic Motion Planningcited by 10
  6. 2022Noisy Symbolic Abstractions for Deep RL: A case study with Reward Machinescited by 10
  7. 2022Skill Machines: Temporal Logic Skill Composition in Reinforcement Learningcited by 9
  8. 2022Deep reinforcement learning under signal temporal logic constraints using Lagrangian relaxationcited by 8
  9. 2025One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learningcited by 8
  10. 2024Neural Reward Machinescited by 7
  11. 2024Directed Exploration in Reinforcement Learning from Linear Temporal Logiccited by 7
  12. 2024Diffusion Meets Options: Hierarchical Generative Skill Composition for Temporally-Extended Taskscited by 7
  13. 2023Exploiting Contextual Structure to Generate Useful Auxiliary Taskscited by 6
  14. 2025Provably Correct Automata Embeddings for Optimal Automata-Conditioned Reinforcement Learningcited by 6
  15. 2025TGPO: Temporal Grounded Policy Optimization for Signal Temporal Logic Taskscited by 6
  16. 2022Categorical semantics of compositional reinforcement learningcited by 5
  17. 2023Reinforcement Learning with Knowledge Representation and Reasoning: A Brief Surveycited by 5
  18. 2025Ground-Compose-Reinforce: Grounding Language in Agentic Behaviours using Limited Datacited by 5
  19. 2022Challenges to Solving Combinatorially Hard Long-Horizon Deep RL Taskscited by 4
  20. 2023Robust Subtask Learning for Compositional Generalizationcited by 4
  21. 2024Planning with a Learned Policy Basis to Optimally Solve Complex Taskscited by 4
  22. 2024Guiding Multi-agent Multi-task Reinforcement Learning by a Hierarchical Framework with Logical Reward Shapingcited by 4
  23. 2023Simple Embodied Language Learning as a Byproduct of Meta-Reinforcement Learningcited by 3
  24. 2024Inductive Generalization in Reinforcement Learning from Specificationscited by 3
  25. 2024Exploiting Hybrid Policy in Reinforcement Learning for Interpretable Temporal Logic Manipulationcited by 3
  26. 2025Learning Reward Machines from Partially Observed Policiescited by 3
  27. 2025Beyond Sliding Windows: Learning to Manage Memory in Non-Markovian Environmentscited by 3
  28. 2026Formal Methods Meet LLMs: Auditing, Monitoring, and Intervention for Compliance of Advanced AI Systemscited by 3
  29. 2022Leveraging class abstraction for commonsense reinforcement learning via residual policy gradient methodscited by 2
  30. 2022Safety-Constrained Policy Transfer with Successor Featurescited by 2
  31. 2025Automaton Constrained Q-Learningcited by 2
  32. 2025Beyond Fixed Tasks: Unsupervised Environment Design for Task-Level Pairscited by 2
  33. 2026PlatoLTL: Scaling LTL-Guided Multi-Task RLcited by 2
  34. 2026Semantically Labelled Automata for Multi-Task Reinforcement Learning with LTL Instructionscited by 2
  35. 2026WeaveLA: Event Driven Cross-Subtask Latent Memory Weaving for Repetitive Robot Manipulationcited by 2
  36. 2022Goal-Conditioned Q-Learning as Knowledge Distillationcited by 1
  37. 2022Generalizing LTL Instructions via Future Dependent Optionscited by 1
  38. 2022Neural Controller Synthesis for Signal Temporal Logic Specifications Using Encoder-Decoder Structured Networkscited by 1

50 most cited of 74 All 74 (JSON)