mlsanityData to 14 Oct 2026

Adventurer: Exploration with BiGAN for Deep Reinforcement Learning

arXiv 2503.18612 ↗First posted 24 Mar 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Recent developments in deep reinforcement learning have been very successful in learning complex, previously intractable problems. Sample efficiency and local optimality, however, remain significant challenges. To address these challenges, novelty-driven exploration strategies have emerged and shown promising potential. Unfortunately, no single algorithm outperforms all others in all tasks and most of them struggle with tasks with high-dimensional and complex observations. In this work, we propose Adventurer, a novelty-driven exploration algorithm that is based on Bidirectional Generative Adversarial Networks (BiGAN), where BiGAN is trained to estimate state novelty. Intuitively, a generator that has been trained on the distribution of visited states should only be able to generate a state coming from the distribution of visited states. As a result, novel states using the generator to reconstruct input states from certain latent representations would lead to larger reconstruction errors. We show that BiGAN performs well in estimating state novelty for complex observations. This novelty estimation method can be combined with intrinsic-reward-based exploration. Our empirical results show that Adventurer produces competitive results on a range of popular benchmark tasks, including continuous robotic manipulation tasks (e.g. Mujoco robotics) and high-dimensional image-based tasks (e.g. Atari games).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 30 of its 51 references that are in the map. First posted on arXiv 24 Mar 2025.

Builds on

30 of 51 references.

References · 30

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2017Hindsight Experience Replaycited by 1,136
  5. 2018Exploration by Random Network Distillationcited by 993
  6. 2016Conditional Image Generation with PixelCNN Decoderscited by 862
  7. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  8. 2017Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discoverycited by 566
  9. 2016Adversarial Feature Learningcited by 350
  10. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
  11. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 303
  12. 2019Challenges of Real-World Reinforcement Learningcited by 291
18 more
  1. 2020First return, then explorecited by 266
  2. 2018Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoderscited by 264
  3. 2017Count-Based Exploration with Neural Density Modelscited by 257
  4. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  5. 2020Never Give Up: Learning Directed Exploration Strategiescited by 212
  6. 2020Deep Learning for Anomaly Detection: A Reviewcited by 159
  7. 2019IPO: Interior-point Policy Optimization under Constraintscited by 132
  8. 2018Efficient GAN-Based Anomaly Detectioncited by 122
  9. 2018Adversarially Learned Anomaly Detectioncited by 115
  10. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 58
  11. 2017EX2: Exploration with Exemplar Models for Deep Reinforcement Learningcited by 42
  12. 2021On Bonus-Based Exploration Methods in the Arcade Learning Environmentcited by 37
  13. 2019A Survey on GANs for Anomaly Detectioncited by 32
  14. 2019Curiosity-Driven Experience Prioritization via Density Estimationcited by 27
  15. 2021A survey on Variational Autoencoders from a GreenAI perspectivecited by 22
  16. 2021Focus on Impact: Indoor Exploration with Intrinsic Motivationcited by 9
  17. 2021CLARA: A Constrained Reinforcement Learning Based Resource Allocation Framework for Network Slicingcited by 7
  18. 2022Generative Adversarial Exploration for Reinforcement Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Explore-Go: Leveraging Exploration for Generalisation in Deep Reinforcement Learning
  2. 2020Shaping Rewards for Reinforcement Learning with Imperfect Demonstrations using Generative Models
  3. 2025Search-Based Adversarial Estimates for Improving Sample Efficiency in Off-Policy Reinforcement Learning
  4. 2022Exploration in Deep Reinforcement Learning: A Survey
  5. 2021Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learning
  6. 2025Exploration Behavior of Untrained Policies
  7. 2025β-DQN: Improving Deep Q-Learning By Evolving the Behavior
  8. 2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization