mlsanityData to 7 Oct 2026

How to Stay Curious while Avoiding Noisy TVs using Aleatoric Uncertainty Estimation

arXiv 2102.04399 ↗First posted 8 Feb 2021cs.LG cs.AIcited by 27

Not on the map. Why?

Abstract (arXiv)

Exploration in environments with sparse rewards is difficult for artificial agents. Curiosity driven learning -- using feed-forward prediction errors as intrinsic rewards -- has achieved some success in these scenarios, but fails when faced with action-dependent noise sources. We present aleatoric mapping agents (AMAs), a neuroscience inspired solution modeled on the cholinergic system of the mammalian brain. AMAs aim to explicitly ascertain which dynamics of the environment are unpredictable, regardless of whether those dynamics are induced by the actions of the agent. This is achieved by generating separate forward predictions for the mean and variance of future states and reducing intrinsic rewards for those transitions with high aleatoric variance. We show AMAs are able to effectively circumvent action-dependent stochastic traps that immobilise conventional curiosity driven agents. The code for all experiments presented in this paper is open sourced: http://github.com/self-supervisor/Escaping-Stochastic-Traps-With-Aleatoric-Mapping-Agents.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 21 of its 57 references that are in the map. First posted on arXiv 8 Feb 2021.

Builds on

21 of 57 references.

References · 21

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  7. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  8. 2018Exploration by Random Network Distillationcited by 966
  9. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
  10. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  11. 2016Deep Exploration via Bootstrapped DQNcited by 518
  12. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
9 more
  1. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
  2. 2019Self-Supervised Exploration via Disagreementcited by 263
  3. 2020Never Give Up: Learning Directed Exploration Strategiescited by 210
  4. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 142
  5. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  6. 2018Model-Based Active Explorationcited by 105
  7. 2019Estimating Risk and Uncertainty in Deep Reinforcement Learningcited by 68
  8. 2018Gotta Learn Fast: A New Benchmark for Generalization in RLcited by 59
  9. 2019Epistemic Risk-Sensitive Reinforcement Learningcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 3; 2024: 6; 2025: 8; 2026: 6.

27 papers.

Sort:

By area:

  1. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
  2. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 79
  3. 2023A Survey on Uncertainty Quantification Methods for Deep Learningcited by 78
  4. 2022A Review of Uncertainty for Deep Reinforcement Learningcited by 36
  5. 2024Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learningcited by 21
  6. 2022General Intelligence Requires Rethinking Explorationcited by 20
  7. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  8. 2024NAVIX: Scaling MiniGrid Environments with JAXcited by 16
  9. 2023Approximate information state based convergence analysis of recurrent Q-learningcited by 5
  10. 2025Uncertainty Prioritized Experience Replaycited by 5
  11. 2022Curiosity in Hindsight: Intrinsic Exploration in Stochastic Environmentscited by 4
  12. 2025Wonder Wins Ways: Curiosity-Driven Exploration through Multi-Agent Contextual Calibrationcited by 4
15 more
  1. 2024A unified uncertainty-aware exploration: Combining epistemic and aleatory uncertaintycited by 2
  2. 2025TW-CRL: Time-Weighted Contrastive Reward Learning for Efficient Inverse Reinforcement Learningcited by 2
  3. 2025NBDI: A Simple and Effective Termination Condition for Skill Extraction from Task-Agnostic Demonstrationscited by 1
  4. 2025Disentangling Uncertainties by Learning Compressed Data Representationcited by 1
  5. 2025ScreenExplorer: Training a Vision-Language Model for Diverse Exploration in Open GUI Worldcited by 1
  6. 2025Exploration by Random Reward Perturbationcited by 1
  7. 2025Beyond Noisy-TVs: Noise-Robust Exploration Via Learning Progress Monitoringcited by 1
  8. 2026DreamSAC: Learning Hamiltonian World Models via Symmetry Explorationcited by 1
  9. 2024Generalized Gaussian Temporal Difference Error for Uncertainty-aware Reinforcement Learningcited by 0
  10. 2024Centralized Reward Agent for Knowledge Sharing and Transfer in Multi-Task Reinforcement Learningcited by 0
  11. 2026Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Explorationcited by 0
  12. 2026CIG: Exploration via Conditional Information Gaincited by 0
  13. 2026Principled Direction-Free Intrinsic Motivation through Model-Free Epistemic Free-Energy Estimatorscited by 0
  14. 2026Risk and Anomaly Identification for Distribution Network Optimal Operation Based on Reinforcement Learning and Uncertainty Quantificationcited by 0
  15. 2026Escaping Local Views: Discovering Latent Concepts for Interpretable Multi-Agent Reinforcement Learningcited by 0