mlsanityData to 7 Oct 2026

The Logical Options Framework

arXiv 2102.12571 ↗First posted 24 Feb 2021cs.AI cs.LG cs.ROcited by 24

Not on the map. Why?

Abstract (arXiv)

Learning composable policies for environments with complex rules and tasks is a challenging problem. We introduce a hierarchical reinforcement learning framework called the Logical Options Framework (LOF) that learns policies that are satisfying, optimal, and composable. LOF efficiently learns policies that satisfy tasks by representing the task as an automaton and integrating it into learning and planning. We provide and prove conditions under which LOF will learn satisfying, optimal policies. And lastly, we show how LOF's learned policies can be composed to satisfy unseen tasks with only 10-50 retraining steps. We evaluate LOF on four tasks in discrete and continuous domains, including a 3D pick-and-place environment.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 19 of its 43 references that are in the map. First posted on arXiv 24 Feb 2021.

Builds on

19 of 43 references.

References · 19

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  3. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  4. 2016Reinforcement Learning With Temporal Logic Rewardscited by 114
  5. 2018SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planningcited by 77
  6. 2018Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specificationscited by 69
  7. 2020A Composable Specification Language for Reinforcement Learning Taskscited by 68
  8. 2017PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planningcited by 64
  9. 2018Learning Actionable Representations with Goal-Conditioned Policiescited by 53
  10. 2018Logically-Constrained Reinforcement Learningcited by 50
  11. 2019PyRep: Bringing V-REP to Deep Robot Learningcited by 43
  12. 2020Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulascited by 36
7 more
  1. 2019Modular Deep Reinforcement Learning with Temporal Logic Specificationscited by 25
  2. 2020Systematic Generalisation through Task Temporal Logic and Deep Reinforcement Learningcited by 21
  3. 2017Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environmentscited by 19
  4. 2019Planning With Uncertain Specifications (PUnS)cited by 11
  5. 2019What Should I Do Now? Marrying Reinforcement Learning and Symbolic Planningcited by 9
  6. 2019Planning with State Abstractions for Non-Markovian Task Specificationscited by 8
  7. 2020A Survey of Knowledge-based Sequential Decision Making under Uncertaintycited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 7; 2023: 1; 2024: 4; 2025: 5; 2026: 6.

24 papers.

Sort:

By area:

  1. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 20
  2. 2021In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specificationscited by 17
  3. 2023Constraint-Conditioned Policy Optimization for Versatile Safe Reinforcement Learningcited by 17
  4. 2025Skill Expansion and Composition in Parameter Spacecited by 13
  5. 2022Skill Machines: Temporal Logic Skill Composition in Reinforcement Learningcited by 7
  6. 2022Learning Task Automata for Reinforcement Learning using Hidden Markov Modelscited by 7
  7. 2022Categorical semantics of compositional reinforcement learningcited by 5
  8. 2025Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learningcited by 5
  9. 2022Hierarchies of Reward Machinescited by 4
  10. 2024Planning with a Learned Policy Basis to Optimally Solve Complex Taskscited by 4
  11. 2024FORM: Learning Expressive and Transferable First-Order Logic Reward Machinescited by 4
  12. 2025Learning Reward Machines from Partially Observed Policiescited by 3
12 more
  1. 2022Hierarchical Reinforcement Learning with AI Planning Modelscited by 2
  2. 2022Safety-Constrained Policy Transfer with Successor Featurescited by 2
  3. 2022Generalizing LTL Instructions via Future Dependent Optionscited by 1
  4. 2024Generalization of Compositional Tasks with Logical Specification via Implicit Planningcited by 1
  5. 2026Grounding LTL Tasks in Sub-Symbolic RL Environments for Zero-Shot Generalizationcited by 1
  6. 2026Zero-Shot Instruction Following in RL via Structured LTL Representationscited by 1
  7. 2026BXRL: Behavior-Explainable Reinforcement Learningcited by 1
  8. 2026SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learningcited by 1
  9. 2025Policy Search, Retrieval, and Composition via Task Similarity in Collaborative Agentic Systemscited by 0
  10. 2025Zero-Shot Instruction Following in RL via Structured LTL Representationscited by 0
  11. 2026Do It for HER: First-Order Temporal Logic Reward Specification in Reinforcement Learning (Extended Version)cited by 0
  12. 2026Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RLcited by 0