mlsanityData to 14 Oct 2026

More Efficient Exploration with Symbolic Priors on Action Sequence Equivalences

arXiv 2110.10632 ↗First posted 20 Oct 2021cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Incorporating prior knowledge in reinforcement learning algorithms is mainly an open question. Even when insights about the environment dynamics are available, reinforcement learning is traditionally used in a tabula rasa setting and must explore and learn everything from scratch. In this paper, we consider the problem of exploiting priors about action sequence equivalence: that is, when different sequences of actions produce the same effect. We propose a new local exploration strategy calibrated to minimize collisions and maximize new state visitations. We show that this strategy can be computed at little cost, by solving a convex optimization problem. By replacing the usual epsilon-greedy strategy in a DQN, we demonstrate its potential in several environments with various dynamic structures.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 48 references that are in the map. First posted on arXiv 20 Oct 2021.

Builds on

25 of 48 references.

References · 25

  1. 2016Progressive Neural Networkscited by 1,361
  2. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  3. 2018Exploration by Random Network Distillationcited by 993
  4. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 641
  5. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  6. 2017Count-Based Exploration with Neural Density Modelscited by 257
  7. 2018Provably Efficient Maximum Entropy Explorationcited by 242
  8. 2020Never Give Up: Learning Directed Exploration Strategiescited by 212
  9. 2019Efficient Exploration via State Marginal Matchingcited by 188
  10. 2017A Rewriting System for Convex Optimization Problemscited by 179
  11. 2015Deep Reinforcement Learning in Large Discrete Action Spacescited by 161
  12. 2020MDP Homomorphic Networks: Group Symmetries in Reinforcement Learningcited by 142
13 more
  1. 2017Action Branching Architectures for Deep Reinforcement Learningcited by 92
  2. 2019Learning Action Representations for Reinforcement Learningcited by 75
  3. 2021Adversarially Guided Actor-Criticcited by 47
  4. 2018Investigating Human Priors for Playing Video Gamescited by 39
  5. 2021Geometric Entropic Explorationcited by 32
  6. 2019Growing Action Spacescited by 29
  7. 2019The Natural Language of Actionscited by 28
  8. 2021There Is No Turning Back: A Self-Supervised Approach for Reversibility-Aware Reinforcement Learningcited by 18
  9. 2019Is Deep Reinforcement Learning Really Superhuman on Atari? Leveling the playing fieldcited by 15
  10. 2017Symmetry Learning for Function Approximation in Reinforcement Learningcited by 14
  11. 2018Online Abstraction with MDP Homomorphisms for Deep Learningcited by 13
  12. 2020Monte-Carlo Graph Search for AlphaZerocited by 7
  13. 2020On the Sensory Commutativity of Action Sequences for Embodied Agentscited by 3

Built on by

Not cited yet.