mlsanityData to 14 Oct 2026

An Information-Geometric Approach to Artificial Curiosity

arXiv 2504.06355 ↗First posted 8 Apr 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Learning in environments with sparse rewards remains a fundamental challenge in reinforcement learning. Artificial curiosity addresses this limitation through intrinsic rewards to guide exploration, however, the precise formulation of these rewards has remained elusive. Ideally, such rewards should depend on the agent's information about the environment, remaining agnostic to its representation -- an invariance central to information geometry. Leveraging this, we show that information monotonicity and invariance under the agent-environment interaction uniquely constrains intrinsic rewards to strictly concave functions of the reciprocal occupancy. Requiring these rewards to yield a principled exploration-exploitation trade-off, via information geodesic interpolation on the occupancy manifold, effectively limits the candidates to those determined by a scalar parameter. Remarkably, special values of this parameter are found to correspond to count-based and maximum entropy exploration. This framework provides important constraints to the engineering of intrinsic rewards while integrating foundational exploration methods into a single, cohesive model.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 16 of its 55 references that are in the map. First posted on arXiv 8 Apr 2025.

Builds on

16 of 55 references.

References · 16

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  6. 2018Exploration by Random Network Distillationcited by 993
  7. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 641
  8. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  9. 2018Provably Efficient Maximum Entropy Explorationcited by 242
  10. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 174
  11. 2020Task-Agnostic Exploration via Policy Gradient of a Non-Parametric State Entropy Estimatecited by 21
  12. 2020Steady State Analysis of Episodic Reinforcement Learningcited by 21
4 more
  1. 2022R'enyi State Entropy for Exploration Acceleration in Reinforcement Learningcited by 18
  2. 2022k-Means Maximum Entropy Explorationcited by 11
  3. 2018Disciplined Geometric Programmingcited by 8
  4. 2024Explosive neural networks via higher-order interactions in curved statistical manifoldscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Intrinsic Rewards for Exploration without Harm from Observational Noise: A Simulation Study Based on the Free Energy Principle
  2. 2026Principled Direction-Free Intrinsic Motivation through Model-Free Epistemic Free-Energy Estimators
  3. 2021Information is Power: Intrinsic Control via Information Capture
  4. 2026Endogenous Exploration in Reinforcement Learning with Intrinsic Curiosity
  5. 2026When Do Intrinsic Rewards Lead to Exploration?
  6. 2025Wonder Wins Ways: Curiosity-Driven Exploration through Multi-Agent Contextual Calibration
  7. 2026Explore Beyond the Boundary Using Entropic Information
  8. 2025On Efficient Bayesian Exploration in Model-Based Reinforcement Learning