mlsanityData to 14 Oct 2026

Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning

arXiv 2310.00229 ↗First posted 30 Sep 2023cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Inspired by human conscious planning, we propose Skipper, a model-based reinforcement learning framework utilizing spatio-temporal abstractions to generalize better in novel situations. It automatically decomposes the given task into smaller, more manageable subtasks, and thus enables sparse decision-making and focused computation on the relevant parts of the environment. The decomposition relies on the extraction of an abstracted proxy problem represented as a directed graph, in which vertices and edges are learned end-to-end from hindsight. Our theoretical analyses provide performance guarantees under appropriate assumptions and establish where our approach is expected to be helpful. Generalization-focused experiments validate Skipper's significant advantage in zero-shot generalization, compared to some existing state-of-the-art hierarchical planning methods.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 45 of its 70 references that are in the map. First posted on arXiv 30 Sep 2023.

Builds on

45 of 70 references.

References · 45

  1. 2016Layer Normalizationcited by 5,669
  2. 2017Neural Discrete Representation Learningcited by 4,712
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  5. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  6. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  8. 2023Mastering Diverse Domains through World Modelscited by 1,253
  9. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  10. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
  11. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  12. 2016The Option-Critic Architecturecited by 448
33 more
  1. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  2. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
  3. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  4. 2018Visual Reinforcement Learning with Imagined Goalscited by 251
  5. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 244
  6. 2018Semi-parametric Topological Memory for Navigationcited by 209
  7. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 194
  8. 2019Multiple Futures Predictioncited by 178
  9. 2019Planning with Goal-Conditioned Policiescited by 149
  10. 2017The Consciousness Priorcited by 107
  11. 2020Invariant Causal Prediction for Block MDPscited by 105
  12. 2019Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneckcited by 101
  13. 2022Deep Hierarchical Planning from Pixelscited by 101
  14. 2018Temporal Difference Models: Model-Free Deep RL for Model-Based Controlcited by 89
  15. 2019Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generationcited by 83
  16. 2019Towards Interpretable Reinforcement Learning Using Attention Augmented Agentscited by 82
  17. 2020World Model as a Graph: Learning Latent Landmarks for Planningcited by 69
  18. 2018How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experimentscited by 60
  19. 2021Memory-efficient Transformers via Top-k Attentioncited by 60
  20. 2021Learning Task Informed Abstractionscited by 58
  21. 2022Skill-based Model-based Reinforcement Learningcited by 50
  22. 2020Neuroevolution of Self-Interpretable Agentscited by 49
  23. 2021Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learningcited by 47
  24. 2020Self-supervised Visual Reinforcement Learning with Object-centric Representationscited by 44
  25. 2019Locality and compositionality in zero-shot learningcited by 29
  26. 2021A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learningcited by 27
  27. 2021Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoningcited by 26
  28. 2021Subgoal Search For Complex Reasoning Taskscited by 21
  29. 2019Reinforcement Learning with Attention that Works: A Self-Supervised Approachcited by 19
  30. 2020Latent Skill Planning for Exploration and Transfercited by 19
  31. 2022Reward-Respecting Subtasks for Model-Based Reinforcement Learningcited by 16
  32. 2022A Look at Value-Based Decision-Time vs. Background Planning Methods Across Different Settingscited by 5
  33. 2023Minimal Value-Equivalent Partial Models for Scalable and Robust Planning in Lifelong Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2024Rejecting Hallucinated State Targets during Planningcited by 1
  2. 2025Learning with Expert Abstractions for Efficient Multi-Task Continuous Controlcited by 0
  3. 2025Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contextscited by 0
  4. 2026Efficient Planning in Reinforcement Learning via Model Introspectioncited by 0