# Active Reinforcement Learning: Observing Rewards at a Cost

arXiv: https://arxiv.org/abs/2011.06709 · HTML: https://mlsanity.com/paper/2011.06709/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Nov 2020.

## Built on by (most cited first, 20 of 20)

- [Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges](https://mlsanity.com/paper/2102.01168/) (2021)
- [Reinforcement Learning from Human Feedback with Active Queries](https://mlsanity.com/paper/2402.09401/) (2024)
- [Active Offline Policy Selection](https://mlsanity.com/paper/2106.10251/) (2021)
- [Monitored Markov Decision Processes](https://mlsanity.com/paper/2402.06819/) (2024)
- [Embodied Active Learning of Relational State Abstractions for Bilevel Planning](https://mlsanity.com/paper/2303.04912/) (2023)
- [Batch Active Learning of Reward Functions from Human Preferences](https://mlsanity.com/paper/2402.15757/) (2024)
- [Beyond Optimism: Exploration With Partially Observable Rewards](https://mlsanity.com/paper/2406.13909/) (2024)
- [Active Advantage-Aligned Online Reinforcement Learning with Offline Data](https://mlsanity.com/paper/2502.07937/) (2025)
- [Scientific Discovery and the Cost of Measurement -- Balancing Information and Cost in Reinforcement Learning](https://mlsanity.com/paper/2112.07535/) (2021)
- [Active Learning for Video Classification with Frame Level Queries](https://mlsanity.com/paper/2307.05587/) (2023)
- [Learning Computational Efficient Bots with Costly Features](https://mlsanity.com/paper/2308.09629/) (2023)
- [Efficient Reinforcement Learning from Human Feedback via Bayesian Preference Inference](https://mlsanity.com/paper/2511.04286/) (2025)
- [Model-Based Exploration in Monitored Markov Decision Processes](https://mlsanity.com/paper/2502.16772/) (2025)
- [Active Measuring in Reinforcement Learning With Delayed Negative Effects](https://mlsanity.com/paper/2510.14315/) (2025)
- [Active Vision Reinforcement Learning under Limited Visual Observability](https://mlsanity.com/paper/2306.00975/) (2023)
- [Which Rewards Matter? Reward Selection for Reinforcement Learning under Limited Feedback](https://mlsanity.com/paper/2510.00144/) (2025)
- [Optimizing Algorithms for Mobile Health Interventions with Active Querying Optimization](https://mlsanity.com/paper/2512.08950/) (2025)
- [Learning to Wait: Synchronizing Agents with the Physical World](https://mlsanity.com/paper/2512.16262/) (2025)
- [Remote Action Generation: Remote Control with Minimal Communication](https://mlsanity.com/paper/2605.01833/) (2026)
- [GoStop: Reinforcement Learning for Adaptive Temporal Aggregation in Event-Based Feature Tracking](https://mlsanity.com/paper/2607.15699/) (2026)

## Abstract (arXiv)

Active reinforcement learning (ARL) is a variant on reinforcement learning where the agent does not observe the reward unless it chooses to pay a query cost c > 0. The central question of ARL is how to quantify the long-term value of reward information. Even in multi-armed bandits, computing the value of this information is intractable and we have to rely on heuristics. We propose and evaluate several heuristic approaches for ARL in multi-armed bandits and (tabular) Markov decision processes, and discuss and illustrate some challenging aspects of the ARL problem.