mlsanityData to 7 Oct 2026

Fast exploration and learning of latent graphs with aliased observations

arXiv 2303.07397 ↗First posted 13 Mar 2023cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We consider the problem of recovering a latent graph where the observations at each node are aliased, and transitions are stochastic. Observations are gathered by an agent traversing the graph. Aliasing means that multiple nodes emit the same observation, so the agent can not know in which node it is located. The agent needs to uncover the hidden topology as accurately as possible and in as few steps as possible. This is equivalent to efficient recovery of the transition probabilities of a partially observable Markov decision process (POMDP) in which the observation probabilities are known. An algorithm for efficiently exploring (and ultimately recovering) the latent graph is provided. Our approach is exponentially faster than naive exploration in a variety of challenging topologies with aliased observations while remaining competitive with existing baselines in the unaliased regime.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 69 references that are in the map. First posted on arXiv 13 Mar 2023.

Builds on

34 of 69 references.

References · 34

  1. 2020Mastering Atari with Discrete World Modelscited by 990
  2. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  3. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  4. 2016Deep Exploration via Bootstrapped DQNcited by 518
  5. 2020Learning to Explore using Active Neural SLAMcited by 308
  6. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  7. 2020First return, then explorecited by 256
  8. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
  9. 2017Cognitive Mapping and Planning for Visual Navigationcited by 209
  10. 2018Semi-parametric Topological Memory for Navigationcited by 203
  11. 2018Deep Variational Reinforcement Learning for POMDPscited by 122
  12. 2019Kinematic State Abstraction and Provably Efficient Rich-Observation Reinforcement Learningcited by 114
22 more
  1. 2017A Laplacian Framework for Option Discovery in Reinforcement Learningcited by 108
  2. 2018Model-Based Active Explorationcited by 105
  3. 2017Neural Map: Structured Memory for Deep Reinforcement Learningcited by 88
  4. 2017Eigenoption Discovery through the Deep Successor Representationcited by 85
  5. 2016DeepMind Labcited by 74
  6. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
  7. 2019Shaping Belief States with Generative Environment Models for RLcited by 65
  8. 2017QMDP-Net: Deep Learning for Planning under Partial Observabilitycited by 51
  9. 2022Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning Approachcited by 51
  10. 2019Variational Recurrent Models for Solving Partially Observable Control Taskscited by 46
  11. 2019Mapping State Space using Landmarks for Universal Goal Reachingcited by 44
  12. 2022Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Modelscited by 42
  13. 2022Evaluating Long-Term Memory in 3D Mazescited by 41
  14. 2013Solving POMDPs by Searching the Space of Finite Policiescited by 25
  15. 2013Incremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processescited by 24
  16. 2020Rapid Task-Solving in Novel Environmentscited by 23
  17. 2015Representing higher-order dependencies in networkscited by 10
  18. 2018Benefits of over-parameterization with EMcited by 10
  19. 2017Learning Overcomplete HMMscited by 9
  20. 2021Map Induction: Compositional spatial submap learning for efficient exploration in novel environmentscited by 9
  21. 2019Learning Transferable Graph Explorationcited by 8
  22. 2013Approximate Planning for Factored POMDPs using Belief State Simplificationcited by 3

Built on by

Not cited yet.