mlsanityData to 7 Oct 2026

A Boolean Task Algebra for Reinforcement Learning

arXiv 2001.01394 ↗First posted 6 Jan 2020cs.LG stat.MLcited by 46

Not on the map. Why?

Abstract (arXiv)

The ability to compose learned skills to solve new tasks is an important property of lifelong-learning agents. In this work, we formalise the logical composition of tasks as a Boolean algebra. This allows us to formulate new tasks in terms of the negation, disjunction and conjunction of a set of base tasks. We then show that by learning goal-oriented value functions and restricting the transition dynamics of the tasks, an agent can solve these new tasks with no further learning. We prove that by composing these value functions in specific ways, we immediately recover the optimal policies for all tasks expressible under the Boolean algebra. We verify our approach in two domains---including a high-dimensional video game environment requiring function approximation---where an agent first learns a set of base skills, and then composes them to solve a super-exponential number of new tasks.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 17; 2023: 9; 2024: 3; 2025: 6; 2026: 3.

46 papers.

Sort:

By area:

  1. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  2. 2021On the Expressivity of Markov Rewardcited by 71
  3. 2021A Survey on Interpretable Reinforcement Learningcited by 70
  4. 2022MoCoDA: Model-based Counterfactual Data Augmentationcited by 44
  5. 2022Modular Lifelong Reinforcement Learning via Neural Compositioncited by 39
  6. 2022Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimalitycited by 35
  7. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  8. 2022Optimistic Linear Support and Successor Features as a Basis for Optimal Policy Transfercited by 22
  9. 2022CompoSuite: A Compositional Reinforcement Learning Benchmarkcited by 21
  10. 2023Constraint-Conditioned Policy Optimization for Versatile Safe Reinforcement Learningcited by 17
  11. 2021Composable Energy Policies for Reactive Motion Generation and Reinforcement Learningcited by 16
  12. 2025A Survey of Continual Reinforcement Learningcited by 16
34 more
  1. 2023Diversifying AI: Towards Creative Chess with AlphaZerocited by 15
  2. 2021Constructing a Good Behavior Basis for Transfer using Generalized Policy Updatescited by 14
  3. 2022Utilizing Prior Solutions for Reward Shaping and Composition in Entropy-Regularized Reinforcement Learningcited by 8
  4. 2023Consistent Aggregation of Objectives with Diverse Time Preferences Requires Non-Markovian Rewardscited by 8
  5. 2025One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learningcited by 8
  6. 2022Skill Machines: Temporal Logic Skill Composition in Reinforcement Learningcited by 7
  7. 2021Globally Optimal Hierarchical Reinforcement Learning for Linearly-Solvable Markov Decision Processescited by 6
  8. 2022Can Interpretable Reinforcement Learning Manage Prosperity Your Way?cited by 6
  9. 2021Lifelong Reinforcement Learning with Temporal Logic Formulas and Reward Machinescited by 5
  10. 2022Categorical semantics of compositional reinforcement learningcited by 5
  11. 2023Reinforcement Learning with Knowledge Representation and Reasoning: A Brief Surveycited by 5
  12. 2023Robotic Manipulation Datasets for Offline Compositional Reinforcement Learningcited by 5
  13. 2025Ground-Compose-Reinforce: Grounding Language in Agentic Behaviours using Limited Datacited by 5
  14. 2022Hierarchies of Reward Machinescited by 4
  15. 2023Hierarchically Composing Level Generators for the Creation of Complex Structurescited by 4
  16. 2024Planning with a Learned Policy Basis to Optimally Solve Complex Taskscited by 4
  17. 2021Learning to Follow Language Instructions with Compositional Policiescited by 3
  18. 2022A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challengescited by 3
  19. 2023Bounding the Optimal Value Function in Compositional Reinforcement Learningcited by 3
  20. 2022World Value Functions: Knowledge Representation for Multitask Reinforcement Learningcited by 2
  21. 2022Safety-Constrained Policy Transfer with Successor Featurescited by 2
  22. 2024XRL-Bench: A Benchmark for Evaluating and Comparing Explainable Reinforcement Learning Techniquescited by 2
  23. 2024Reduce, Reuse, Recycle: Categories for Compositional Reinforcement Learningcited by 2
  24. 2022Operator Deep Q-Learning: Zero-Shot Reward Transferring in Reinforcement Learningcited by 1
  25. 2022Reinforcement Learning with Intrinsic Affinity for Personalized Prosperity Managementcited by 1
  26. 2022World Value Functions: Knowledge Representation for Learning and Planningcited by 1
  27. 2023Leveraging Prior Knowledge in Reinforcement Learning via Double-Sided Bounds on the Value Functioncited by 0
  28. 2023Safety-Aware Task Composition for Discrete and Continuous Reinforcement Learningcited by 0
  29. 2025Compositional Instruction Following with Language Models and Reinforcement Learningcited by 0
  30. 2025Realizable Abstractions: Near-Optimal Hierarchical Reinforcement Learningcited by 0
  31. 2025ARC: Leveraging Compositional Representations for Cross-Problem Learning on VRPscited by 0
  32. 2026Task diversity produces systematic transfer but inhibits continual reinforcement learningcited by 0
  33. 2026A Goal-Set Characterization of Task Composition in the Boolean Task Algebracited by 0
  34. 2026The syntax and semantics of goalscited by 0