mlsanityData to 7 Oct 2026

A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learning

arXiv 2106.02097 ↗First posted 3 Jun 2021cs.AI cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

We present an end-to-end, model-based deep reinforcement learning agent which dynamically attends to relevant parts of its state during planning. The agent uses a bottleneck mechanism over a set-based representation to force the number of entities to which the agent attends at each planning step to be small. In experiments, we investigate the bottleneck mechanism with several sets of customized environments featuring different challenges. We consistently observe that the design allows the planning agents to generalize their learned task-solving abilities in compatible unseen environments by attending to the relevant objects, leading to better out-of-distribution generalization performance.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 36 of its 65 references that are in the map. First posted on arXiv 3 Jun 2021.

Builds on

36 of 65 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2016Layer Normalizationcited by 5,579
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  6. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  7. 2018World Modelscited by 1,444
  8. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  9. 2017Deep Setscited by 1,394
  10. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  11. 2020Mastering Atari with Discrete World Modelscited by 990
  12. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
24 more
  1. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  2. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  3. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  4. 2019Model-Based Reinforcement Learning for Ataricited by 495
  5. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
  6. 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 273
  7. 2018Distributed Prioritized Experience Replaycited by 260
  8. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  9. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 220
  10. 2020How Neural Networks Extrapolate: From Feedforward to Graph Neural Networkscited by 218
  11. 2019Benchmarking Model-Based Reinforcement Learningcited by 158
  12. 2017The Consciousness Priorcited by 105
  13. 2016The Predictron: End-To-End Learning and Planningcited by 92
  14. 2018Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigationcited by 84
  15. 2021Learning and Planning in Complex Action Spacescited by 76
  16. 2020On the role of planning in model-based deep reinforcement learningcited by 57
  17. 2018Sparse Attentive Backtracking: Temporal CreditAssignment Through Remindingcited by 33
  18. 2020Conditional Set Generation with Transformerscited by 32
  19. 2021Modeling Event Plausibility with Consistent Conceptual Abstractioncited by 20
  20. 2020Learning Object-Centric Video Models by Contrasting Setscited by 19
  21. 2020Refactoring Policy for Compositional Generalizability using Self-Supervised Object Proposalscited by 9
  22. 2020BabyAI 1.1cited by 8
  23. 2020Investigating Simple Object Representations in Model-Free Deep Reinforcement Learningcited by 5
  24. 2020Episodic Memory for Learning Subjective-Timescale Modelscited by 3

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 5; 2023: 7; 2024: 8; 2025: 5.

27 papers.

Sort:

By area:

  1. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
  2. 2025RL Fine-Tuning Heals OOD Forgetting in SFTcited by 27
  3. 2021A Theory of Consciousness from a Theoretical Computer Science Perspective: Insights from the Conscious Turing Machinecited by 22
  4. 2024NAVIX: Scaling MiniGrid Environments with JAXcited by 16
  5. 2022On the link between conscious function and general intelligence in humans and machinescited by 15
  6. 2021Temporally Abstract Partial Modelscited by 11
  7. 2024Dynamic Model Predictive Shielding for Provably Safe Reinforcement Learningcited by 11
  8. 2024SPARTAN: A Sparse Transformer World Model Attending to What Matterscited by 10
  9. 2024Representation Learning on Heterophilic Graph with Directional Neighborhood Attentioncited by 6
  10. 2025Translate Policy to Language: Flow Matching Generated Rewards for LLM Explanationscited by 6
  11. 2022A Look at Value-Based Decision-Time vs. Background Planning Methods Across Different Settingscited by 5
  12. 2022Compositional Generalization in Grounded Language Learning via Induced Model Sparsitycited by 4
15 more
  1. 2023Reusable Slotwise Mechanismscited by 4
  2. 2023Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learningcited by 4
  3. 2025Towards Empowerment Gain through Causal Structure Learning in Model-Based RLcited by 4
  4. 2024Revealing the Pitfalls and Re-Evaluating the Advancement of Heterophilic Graph Learningcited by 3
  5. 2024Flexible Diffusion Scopes with Parameterized Laplacian for Heterophilic Graph Learningcited by 3
  6. 2022Boolean Decision Rules for Reinforcement Learning Policy Summarisationcited by 2
  7. 2023Minimal Value-Equivalent Partial Models for Scalable and Robust Planning in Lifelong Reinforcement Learningcited by 2
  8. 2023A Tractable Inference Perspective of Offline RLcited by 2
  9. 2025Deep Reinforcement Learning via Object-Centric Attentioncited by 2
  10. 2023On Addressing the Limitations of Graph Neural Networkscited by 1
  11. 2024Rejecting Hallucinated State Targets during Planningcited by 1
  12. 2022Reinforcement Learning with Brain-Inspired Modulation can Improve Adaptation to Environmental Changescited by 0
  13. 2023DELTA: Dynamic Embedding Learning with Truncated Conscious Attention for CTR Predictioncited by 0
  14. 2024Partial Models for Building Adaptive Model-Based Reinforcement Learning Agentscited by 0
  15. 2025Inductive Biases for Zero-shot Systematic Generalization in Language-informed Reinforcement Learningcited by 0