# Mega-Reward: Achieving Human-Level Play without Extrinsic Rewards

arXiv: https://arxiv.org/abs/1905.04640 · HTML: https://mlsanity.com/paper/1905.04640/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 May 2019.

## Built on by (most cited first, 5 of 5)

- [Causal Influence Detection for Improving Efficiency in Reinforcement Learning](https://mlsanity.com/paper/2106.03443/) (2021)
- [Mutual Information State Intrinsic Control](https://mlsanity.com/paper/2103.08107/) (2021)
- [Causal Action Influence Aware Counterfactual Data Augmentation](https://mlsanity.com/paper/2405.18917/) (2024)
- [Reinforcement Learning with Exogenous States and Rewards](https://mlsanity.com/paper/2303.12957/) (2023)
- [Did I do that? Blame as a means to identify controlled effects in reinforcement learning](https://mlsanity.com/paper/2106.00266/) (2021)

## Abstract (arXiv)

Intrinsic rewards were introduced to simulate how human intelligence works; they are usually evaluated by intrinsically-motivated play, i.e., playing games without extrinsic rewards but evaluated with extrinsic rewards. However, none of the existing intrinsic reward approaches can achieve human-level performance under this very challenging setting of intrinsically-motivated play. In this work, we propose a novel megalomania-driven intrinsic reward (called mega-reward), which, to our knowledge, is the first approach that achieves human-level performance in intrinsically-motivated play. Intuitively, mega-reward comes from the observation that infants' intelligence develops when they try to gain more control on entities in an environment; therefore, mega-reward aims to maximize the control capabilities of agents on given entities in a given environment. To formalize mega-reward, a relational transition model is proposed to bridge the gaps between direct and latent control. Experimental studies show that mega-reward (i) can greatly outperform all state-of-the-art intrinsic reward approaches, (ii) generally achieves the same level of performance as Ex-PPO and professional human-level scores, and (iii) has also a superior performance when it is incorporated with extrinsic rewards.