# Active Reinforcement Learning over MDPs

arXiv: https://arxiv.org/abs/2108.02323 · HTML: https://mlsanity.com/paper/2108.02323/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 29 references that are in the map.
- First posted on arXiv 5 Aug 2021.

## Builds on (most cited first, 23 of 23)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels](https://mlsanity.com/paper/2004.13649/) (2020)
- [Reinforcement Learning with Augmented Data](https://mlsanity.com/paper/2004.14990/) (2020)
- [Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables](https://mlsanity.com/paper/1903.08254/) (2019)
- [Quantifying Generalization in Reinforcement Learning](https://mlsanity.com/paper/1812.02341/) (2018)
- [First return, then explore](https://mlsanity.com/paper/2004.12919/) (2020)
- [Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning](https://mlsanity.com/paper/1910.05396/) (2019)
- [Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck](https://mlsanity.com/paper/1910.12911/) (2019)
- [Deep Learning for Procedural Content Generation](https://mlsanity.com/paper/2010.04548/) (2020)
- [Towards Generalization and Simplicity in Continuous Control](https://mlsanity.com/paper/1703.02660/) (2017)
- [Increasing Generality in Machine Learning through Procedural Content Generation](https://mlsanity.com/paper/1911.13071/) (2019)
- [Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation](https://mlsanity.com/paper/1806.07377/) (2018)
- [Discount Factor as a Regularizer in Reinforcement Learning](https://mlsanity.com/paper/2007.02040/) (2020)
- [Invariant Policy Optimization: Towards Stronger Generalization in Reinforcement Learning](https://mlsanity.com/paper/2006.01096/) (2020)
- [Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learning](https://mlsanity.com/paper/2010.13303/) (2020)
- [StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer Learning](https://mlsanity.com/paper/1804.00810/) (2018)
- [Rotation, Translation, and Cropping for Zero-Shot Generalization](https://mlsanity.com/paper/2001.09908/) (2020)
- [When Is Generalizable Reinforcement Learning Tractable?](https://mlsanity.com/paper/2101.00300/) (2021)
- [Deep Curiosity Search: Intra-Life Exploration Can Improve Performance on Challenging Deep Reinforcement Learning Problems](https://mlsanity.com/paper/1806.00553/) (2018)

## Abstract (arXiv)

The past decade has seen the rapid development of Reinforcement Learning, which acquires impressive performance with numerous training resources. However, one of the greatest challenges in RL is generalization efficiency (i.e., generalization performance in a unit time). This paper proposes a framework of Active Reinforcement Learning (ARL) over MDPs to improve generalization efficiency in a limited resource by instance selection. Given a number of instances, the algorithm chooses out valuable instances as training sets while training the policy, thereby costing fewer resources. Unlike existing approaches, we attempt to actively select and use training data rather than train on all the given data, thereby costing fewer resources. Furthermore, we introduce a general instance evaluation metrics and selection mechanism into the framework. Experiments results reveal that the proposed framework with Proximal Policy Optimization as policy optimizer can effectively improve generalization efficiency than unselect-ed and unbiased selected methods.