mlsanityData to 14 Oct 2026

EX2: Exploration with Exemplar Models for Deep Reinforcement Learning

arXiv 1703.01260 ↗First posted 3 Mar 2017cs.LGcited by 42

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning algorithms have been shown to learn complex tasks using highly general policy classes. However, sparse reward problems remain a significant challenge. Exploration methods based on novelty detection have been particularly successful in such settings but typically require generative or predictive models of the observations, which can be difficult to train when the observations are very high-dimensional and complex, as in the case of raw images. We propose a novelty detection algorithm for exploration that is based entirely on discriminatively trained exemplar models, where classifiers are trained to discriminate each visited state against all others. Intuitively, novel states are easier to distinguish against other states seen during training. We show that this kind of discriminative modeling corresponds to implicit density estimation, and that it can be combined with count-based exploration to produce competitive results on a range of popular benchmark tasks, including state-of-the-art results on challenging egocentric observations in the vizDoom benchmark.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). First posted on arXiv 3 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 9; 2023: 12; 2024: 6; 2025: 4; 2026: 3.

42 papers.

Sort:

By area:

  1. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  2. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 166
  3. 2022Human-to-Robot Imitation in the Wildcited by 161
  4. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  5. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 73
  6. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 58
  7. 2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57
  8. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 28
  9. 2021MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learningcited by 24
  10. 2024Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learningcited by 22
  11. 2023Model-based Offline Reinforcement Learning with Count-based Conservatismcited by 16
  12. 2023Off-Policy RL Algorithms Can be Sample-Efficient for Continuous Control via Sample Multiple Reusecited by 12
30 more
  1. 2021Information is Power: Intrinsic Control via Information Capturecited by 11
  2. 2025Exploration-Driven Generative Interactive Environmentscited by 9
  3. 2023Novelty Detection in Reinforcement Learning with World Modelscited by 8
  4. 2023Offline Retraining for Online RL: Decoupled Policy Learning to Mitigate Exploration Biascited by 7
  5. 2022From Psychological Curiosity to Artificial Curiosity: Curiosity-Driven Learning in Artificial Intelligence Taskscited by 6
  6. 2023Successor-Predecessor Intrinsic Explorationcited by 6
  7. 2024Constrained Intrinsic Motivation for Reinforcement Learningcited by 6
  8. 2024Random Latent Exploration for Deep Reinforcement Learningcited by 6
  9. 2024Safety through feedback in Constrained RLcited by 5
  10. 2026Emergence of Exploration in Policy Gradient Reinforcement Learning via Retryingcited by 5
  11. 2022Curiosity in Hindsight: Intrinsic Exploration in Stochastic Environmentscited by 4
  12. 2023ALP: Action-Aware Embodied Learning for Perceptioncited by 4
  13. 2023Intrinsically motivated graph exploration using network theories of human curiositycited by 3
  14. 2024Image-Based Deep Reinforcement Learning with Intrinsically Motivated Stimuli: On the Execution of Complex Robotic Taskscited by 3
  15. 2021Playtesting: What is Beyond Personascited by 2
  16. 2025Generative Actor Criticcited by 2
  17. 20263SPO: State-Score-Supervised Policy Optimization for LLM Agentscited by 2
  18. 2021Intrinsically Motivated Self-supervised Learning in Reinforcement Learningcited by 1
  19. 2022Normality-Guided Distributional Reinforcement Learning for Continuous Controlcited by 1
  20. 2022CIM: Constrained Intrinsic Motivation for Sparse-Reward Continuous Controlcited by 1
  21. 2023Centralized Cooperative Exploration Policy for Continuous Control Taskscited by 1
  22. 2023Time-Myopic Go-Explore: Learning A State Representation for the Go-Explore Paradigmcited by 1
  23. 2025Periodic Skill Discoverycited by 1
  24. 2022Automatic Reward Design via Learning Motivation-Consistent Intrinsic Rewardscited by 0
  25. 2022Curiosity creates Diversity in Policy Searchcited by 0
  26. 2023An Autonomous Non-monolithic Agent with Multi-mode Exploration based on Options Frameworkcited by 0
  27. 2023Generative Intrinsic Optimization: Intrinsic Control with Model Learningcited by 0
  28. 2024Deterministic Exploration via Stationary Bellman Error Maximizationcited by 0
  29. 2025Adventurer: Exploration with BiGAN for Deep Reinforcement Learningcited by 0
  30. 2026Escaping Offline Pessimism: Vector-Field Reward Shaping for Safe Frontier Explorationcited by 0