Data to 7 Oct 2026

Large-Scale Study of Curiosity-Driven Learning

arXiv 1808.04355 ↗First posted 13 Aug 2018cs.LG cs.AI cs.CVcited by 277

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning algorithms rely on carefully engineering environment rewards that are extrinsic to the agent. However, annotating each environment with hand-designed, dense rewards is not scalable, motivating the need for developing reward functions that are intrinsic to the agent. Curiosity is a type of intrinsic reward function which uses prediction error as reward signal. In this paper: (a) We perform the first large-scale study of purely curiosity-driven learning, i.e. without any extrinsic rewards, across 54 standard benchmark environments, including the Atari game suite. Our results show surprisingly good performance, and a high degree of alignment between the intrinsic curiosity objective and the hand-designed extrinsic rewards of many game environments. (b) We investigate the effect of using different feature spaces for computing prediction error and show that random features are sufficient for many popular RL game benchmarks, but learned features appear to generalize better (e.g. to novel game levels in Super Mario Bros.). (c) We demonstrate limitations of the prediction-based rewards in stochastic setups. Game-play videos and code are at https://pathak22.github.io/large-scale-curiosity/

Key facts. Cited by 277 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 72; 2022: 61; 2023: 48; 2024: 33; 2025: 40; 2026: 22.

277 papers.

Sort:

By area:

2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
2021Benchmarking the Spectrum of Agent Capabilitiescited by 193
2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
2021APS: Active Pretraining with Successor Featurescited by 109
2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 99
2023CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalizationcited by 88
2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 71
2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
2021Learning Auxiliary Monocular Contexts Helps Monocular 3D Object Detectioncited by 66

38 more

2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66
2023Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraftcited by 60
2021Reinforcement Learning, Bit by Bitcited by 59
2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
2021Continual Learning for Real-World Autonomous Systems: Algorithms, Challenges and Frameworkscited by 56
2022Semantic Exploration from Language Abstractions and Pretrained Representationscited by 51
2022Exploration via Elliptical Episodic Bonusescited by 51
2021Towards Robust Bisimulation Metric Learningcited by 49
2021Episodic Multi-agent Reinforcement Learning with Curiosity-Driven Explorationcited by 49
2024RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulationcited by 48
2021Offline Reinforcement Learning as Anti-Explorationcited by 44
2021Curious Representation Learning for Embodied Intelligencecited by 43
2021ELLA: Exploration through Learned Language Abstractioncited by 42
2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
2021On Bonus-Based Exploration Methods in the Arcade Learning Environmentcited by 37
2020Intrinsic Reward Driven Imitation Learning via Generative Modelcited by 35
2024Exploration and Anti-Exploration with Distributional Random Network Distillationcited by 32
2021Unsupervised Learning of Visual 3D Keypoints for Controlcited by 30
2021Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environmentscited by 28
2021Provable RL with Exogenous Distractors via Multistep Inverse Dynamicscited by 28
2023Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learningcited by 28
2021How to Stay Curious while Avoiding Noisy TVs using Aleatoric Uncertainty Estimationcited by 27
2021A statistical framework for efficient out of distribution detection in deep neural networkscited by 27
2022Semi-Supervised Offline Reinforcement Learning with Action-Free Trajectoriescited by 27
2021Principled Exploration via Optimistic Bootstrapping and Backward Inductioncited by 26
2022The Challenges of Exploration for Offline Reinforcement Learningcited by 26
2021Decoupled Reinforcement Learning to Stabilise Intrinsically-Motivated Explorationcited by 25
2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
2025Enhancing Personalized Multi-Turn Dialogue with Curiosity Rewardcited by 25
2021The MineRL BASALT Competition on Learning from Human Feedbackcited by 24
2022The Free Energy Principle for Perception and Action: A Deep Learning Perspectivecited by 24
2022Redeeming Intrinsic Rewards via Constrained Optimizationcited by 24
2021Curiosity-Driven Exploration via Latent Bayesian Surprisecited by 22
2021Beyond Fine-Tuning: Transferring Behavior in Reinforcement Learningcited by 21
2021Creativity and Machine Learning: A Surveycited by 21
2021Dynamic Bottleneck for Robust Self-Supervised Explorationcited by 21
2024Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learningcited by 21
2022Mastering the Unsupervised Reinforcement Learning Benchmark from Pixelscited by 20

50 most cited of 277 All 277 (JSON)