# Options of Interest: Temporal Abstraction with Interest Functions

arXiv: https://arxiv.org/abs/2001.00271 · HTML: https://mlsanity.com/paper/2001.00271/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jan 2020.

## Built on by (most cited first, 30 of 30)

- [Motif: Intrinsic Motivation from Artificial Intelligence Feedback](https://mlsanity.com/paper/2310.00166/) (2023)
- [Code as Reward: Empowering Reinforcement Learning with VLMs](https://mlsanity.com/paper/2402.04764/) (2024)
- [Deep Laplacian-based Options for Temporally-Extended Exploration](https://mlsanity.com/paper/2301.11181/) (2023)
- [Flexible Option Learning](https://mlsanity.com/paper/2112.03097/) (2021)
- [MaestroMotif: Skill Design from Artificial Intelligence Feedback](https://mlsanity.com/paper/2412.08542/) (2024)
- [Semantic HELM: A Human-Readable Memory for Reinforcement Learning](https://mlsanity.com/paper/2306.09312/) (2023)
- [Temporally Abstract Partial Models](https://mlsanity.com/paper/2108.03213/) (2021)
- [SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions](https://mlsanity.com/paper/2410.18416/) (2024)
- [Context-Specific Representation Abstraction for Deep Option Learning](https://mlsanity.com/paper/2109.09876/) (2021)
- [Attention Option-Critic](https://mlsanity.com/paper/2201.02628/) (2022)
- [Bayesian Nonparametrics for Offline Skill Discovery](https://mlsanity.com/paper/2202.04675/) (2022)
- [Autonomous Option Invention for Continual Hierarchical Reinforcement Learning and Planning](https://mlsanity.com/paper/2412.16395/) (2024)
- [Priors, Hierarchy, and Information Asymmetry for Skill Transfer in Reinforcement Learning](https://mlsanity.com/paper/2201.08115/) (2022)
- [Toward Discovering Options that Achieve Faster Planning](https://mlsanity.com/paper/2205.12515/) (2022)
- [The Paradox of Choice: Using Attention in Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2201.09653/) (2022)
- [Machine Learning Meets Advanced Robotic Manipulation](https://mlsanity.com/paper/2309.12560/) (2023)
- [Toward Human-AI Alignment in Large-Scale Multi-Player Games](https://mlsanity.com/paper/2402.03575/) (2024)
- [Subgoal Discovery Using a Free Energy Paradigm and State Aggregations](https://mlsanity.com/paper/2412.16687/) (2024)
- [Hierarchical Reinforcement Learning in Multi-Goal Spatial Navigation with Autonomous Mobile Robots](https://mlsanity.com/paper/2504.18794/) (2025)
- [Coordinating Policies Among Multiple Agents via an Intelligent Communication Channel](https://mlsanity.com/paper/2205.10607/) (2022)
- [Reusable Options through Gradient-based Meta Learning](https://mlsanity.com/paper/2212.11726/) (2022)
- [Discerning Temporal Difference Learning](https://mlsanity.com/paper/2310.08091/) (2023)
- [Temporal Abstraction in Reinforcement Learning with Offline Data](https://mlsanity.com/paper/2407.15241/) (2024)
- [Learning Temporal Abstractions via Variational Homomorphisms in Option-Induced Abstract MDPs](https://mlsanity.com/paper/2507.16473/) (2025)
- [Realizable Abstractions: Near-Optimal Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2512.04958/) (2025)
- [CRoSS: A Continual Robotic Simulation Suite for Scalable Reinforcement Learning with High Task Diversity and Realistic Physics Simulation](https://mlsanity.com/paper/2602.04868/) (2026)
- [Enabling Option Learning in Sparse Rewards with Hindsight Experience Replay](https://mlsanity.com/paper/2602.13865/) (2026)
- [Learning Multi-Timescale Abstractions for Hierarchical Combinatorial Planning](https://mlsanity.com/paper/2605.17058/) (2026)
- [When Do Options Help? Policy Necrosis and Redundant Coverage in Option-Critic](https://mlsanity.com/paper/2609.05508/) (2026)
- [Up and Down the Abstraction Ladder: Code-Based Skills for Language Agents](https://mlsanity.com/paper/2609.31076/) (2026)

## Abstract (arXiv)

Temporal abstraction refers to the ability of an agent to use behaviours of controllers which act for a limited, variable amount of time. The options framework describes such behaviours as consisting of a subset of states in which they can initiate, an internal policy and a stochastic termination condition. However, much of the subsequent work on option discovery has ignored the initiation set, because of difficulty in learning it from data. We provide a generalization of initiation sets suitable for general function approximation, by defining an interest function associated with an option. We derive a gradient-based learning algorithm for interest functions, leading to a new interest-option-critic architecture. We investigate how interest functions can be leveraged to learn interpretable and reusable temporal abstractions. We demonstrate the efficacy of the proposed approach through quantitative and qualitative results, in both discrete and continuous environments.