mlsanityData to 14 Oct 2026

Temporally Abstract Partial Models

arXiv 2108.03213 ↗First posted 6 Aug 2021cs.LG cs.AI stat.MLcited by 11

Not on the map. Why?

Abstract (arXiv)

Humans and animals have the ability to reason and make predictions about different courses of action at many time scales. In reinforcement learning, option models (Sutton, Precup & Singh, 1999; Precup, 2000) provide the framework for this kind of temporally abstract prediction and reasoning. Natural intelligent agents are also able to focus their attention on courses of action that are relevant or feasible in a given situation, sometimes termed affordable actions. In this paper, we define a notion of affordances for options, and develop temporally abstract partial option models, that take into account the fact that an option might be affordable only in certain situations. We analyze the trade-offs between estimation and approximation error in planning and learning when using such models, and identify some interesting special cases. Additionally, we demonstrate empirically the potential impact of partial option models on the efficiency of planning.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 17 of its 48 references that are in the map. First posted on arXiv 6 Aug 2021.

Builds on

17 of 48 references.

References · 17

  1. 2016The Option-Critic Architecturecited by 448
  2. 2017Value Prediction Networkcited by 117
  3. 2019Shaping Belief States with Generative Environment Models for RLcited by 66
  4. 2017When Waiting is not an Option : Learning Options with a Deliberation Costcited by 58
  5. 2019COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Explorationcited by 57
  6. 2016Self-Correcting Models for Model-Based Reinforcement Learningcited by 43
  7. 2020Deep Affordance Foresight: Planning Through What Can Be Done in the Futurecited by 41
  8. 2018Neural Predictive Belief Representationscited by 40
  9. 2020What can I do here? A Theory of Affordances in Reinforcement Learningcited by 35
  10. 2020Options of Interest: Temporal Abstraction with Interest Functionscited by 30
  11. 2021A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learningcited by 27
  12. 2019The Termination Criticcited by 24
5 more
  1. 2019Lifelong Learning with a Changing Action Setcited by 17
  2. 2021Launchpad: A Programming Model for Distributed Machine Learning Researchcited by 14
  3. 2017What can you do with a rock? Affordance extraction via word embeddingscited by 13
  4. 2018Learning Awareness Modelscited by 13
  5. 2018Multi-modal Feedback for Affordance-driven Interactive Reinforcement Learningcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 4; 2024: 3; 2025: 1; 2026: 1.

11 papers.

  1. 2021Flexible Option Learningcited by 25
  2. 2024Empowering Large Language Models on Robotic Manipulation with Affordance Promptingcited by 17
  3. 2023Recent Advances of Deep Robotic Affordance Learning: A Reinforcement Learning Perspectivecited by 10
  4. 2022The Paradox of Choice: Using Attention in Hierarchical Reinforcement Learningcited by 2
  5. 2023Minimal Value-Equivalent Partial Models for Scalable and Robust Planning in Lifelong Reinforcement Learningcited by 2
  6. 2023Self-Supervised Learning of Action Affordances as Interaction Modescited by 2
  7. 2024Toward Human-AI Alignment in Large-Scale Multi-Player Gamescited by 2
  8. 2026Joint Learning of Hierarchical Neural Options and Abstract World Modelcited by 1
  9. 2023Forecaster: Towards Temporally Abstract Tree-Search Planning from Pixelscited by 0
  10. 2024Partial Models for Building Adaptive Model-Based Reinforcement Learning Agentscited by 0
  11. 2025A Unified Theory of Compositionality, Modularity, and Interpretability in Markov Decision Processescited by 0