mlsanityData to 7 Oct 2026

Efficient Model-Based Deep Reinforcement Learning with Variational State Tabulation

arXiv 1802.04325 ↗First posted 12 Feb 2018cs.LG cs.AI stat.MLcited by 20

Not on the map. Why?

Abstract (arXiv)

Modern reinforcement learning algorithms reach super-human performance on many board and video games, but they are sample inefficient, i.e. they typically require significantly more playing experience than humans to reach an equal performance level. To improve sample efficiency, an agent may build a model of the environment and use planning methods to update its policy. In this article we introduce Variational State Tabulation (VaST), which maps an environment with a high-dimensional state space (e.g. the space of visual inputs) to an abstract tabular model. Prioritized sweeping with small backups, a highly efficient planning method, can then be used to update state-action values. We show how VaST can rapidly learn to maximize reward in tasks like 3D navigation and efficiently adapt to sudden changes in rewards or transition probabilities.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 7; 2022: 4; 2023: 4; 2024: 2; 2025: 1; 2026: 1.

20 papers.

Sort:

By area:

  1. 2021Towards Robust Bisimulation Metric Learningcited by 49
  2. 2021Learning Markov State Abstractions for Deep Reinforcement Learningcited by 42
  3. 2021Proper Value Equivalencecited by 33
  4. 2021Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoningcited by 25
  5. 2020DeepAveragers: Offline Reinforcement Learning by Solving Derived Non-Parametric MDPscited by 16
  6. 2021Hierarchies of Planning and Reinforcement Learning for Robot Navigationcited by 11
  7. 2022LECO: Learnable Episodic Count for Task-Specific Intrinsic Rewardcited by 11
  8. 2021Distillation of RL Policies with Formal Guarantees via Variational Abstraction of Markov Decision Processes (Technical Report)cited by 9
  9. 2024Efficient Multi-agent Reinforcement Learning by Planningcited by 9
  10. 2023What model does MuZero learn?cited by 7
  11. 2022Factored World Models for Zero-Shot Generalization in Robotic Manipulationcited by 6
  12. 2024Learning to Play Atari in a World of Tokenscited by 6
8 more
  1. 2023Predictive World Models from Real-World Partial Observationscited by 4
  2. 2023MABL: Bi-Level Latent-Variable World Model for Sample-Efficient Multi-Agent Reinforcement Learningcited by 4
  3. 2021Hierarchical Representation Learning for Markov Decision Processescited by 3
  4. 2022Visual processing in context of reinforcement learningcited by 0
  5. 2022PRISM: Probabilistic Real-Time Inference in Spatial World Modelscited by 0
  6. 2023Wasserstein Auto-encoded MDPs: Formal Verification of Efficiently Distilled RL Policies with Many-sided Guaranteescited by 0
  7. 2025A Comparative Study of OpenMP Scheduling Algorithm Selection Strategiescited by 0
  8. 2026Spectral Prioritized Sweeping in Nonstationary Reinforcement Learningcited by 0