mlsanityData to 7 Oct 2026

When in Doubt, Think Slow: Iterative Reasoning with Latent Imagination

arXiv 2402.15283 ↗First posted 23 Feb 2024cs.LG cs.AIcited by 0

On the map: World models → World models for language agents

Abstract (arXiv)

In an unfamiliar setting, a model-based reinforcement learning agent can be limited by the accuracy of its world model. In this work, we present a novel, training-free approach to improving the performance of such agents separately from planning and learning. We do so by applying iterative inference at decision-time, to fine-tune the inferred agent states based on the coherence of future state representations. Our approach achieves a consistent improvement in both reconstruction accuracy and task performance when applied to visual 3D navigation tasks. We go on to show that considering more future states further improves the performance of the agent in partially-observable environments, but not in a fully-observable one. Finally, we demonstrate that agents with less training pre-evaluation benefit most from our approach.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 33 of its 62 references that are in the map. First posted on arXiv 23 Feb 2024.

Builds on

33 of 62 references.

References · 33

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  4. 2023Mastering Diverse Domains through World Modelscited by 1,193
  5. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  6. 2018Universal Transformerscited by 613
  7. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  8. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  9. 2017Thinking Fast and Slow with Deep Learning and Tree Searchcited by 245
  10. 2018Provably Efficient Maximum Entropy Explorationcited by 240
  11. 2023SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Taskscited by 166
  12. 2017Adaptive Neural Networks for Efficient Inferencecited by 161
21 more
  1. 2021PonderNet: Learning to Pondercited by 156
  2. 2018Inference Suboptimality in Variational Autoencoderscited by 131
  3. 2021Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networkscited by 108
  4. 2016Hierarchical Multiscale Recurrent Neural Networkscited by 102
  5. 2018Iterative Amortized Inferencecited by 93
  6. 2021Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoningcited by 85
  7. 2018Semi-Amortized Variational Autoencoderscited by 83
  8. 2019A survey on intrinsic motivation in reinforcement learningcited by 79
  9. 2020Deep active inference agents using Monte-Carlo methodscited by 72
  10. 2020On the Relationship Between Active Inference and Control as Inferencecited by 42
  11. 2022Learning Iterative Reasoning through Energy Minimizationcited by 32
  12. 2022Hybrid Predictive Coding: Inferring, Fast and Slowcited by 25
  13. 2017On the challenges of learning with inference networks on sparse, high-dimensional datacited by 21
  14. 2020Iterative Amortized Policy Optimizationcited by 13
  15. 2020Differentiable Adaptive Computation Time for Visual Reasoningcited by 12
  16. 2021Information is Power: Intrinsic Control via Information Capturecited by 11
  17. 2023Sample as You Infer: Predictive Coding With Langevin Dynamicscited by 11
  18. 2020Control as Hybrid Inferencecited by 9
  19. 2021Variational Predictive Routing with Nested Subjective Timescalescited by 9
  20. 2023Adaptive recurrent vision performs zero-shot computation scaling to unseen difficulty levelscited by 6
  21. 2020Reinforcement Learning as Iterative and Amortised Inferencecited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Enhancing data efficiency in reinforcement learning: a novel imagination mechanism based on mesh information propagation
  2. 2026LatentPilot: Scene-Aware Vision-and-Language Navigation by Dreaming Ahead with Latent Visual Reasoning
  3. 2025Reinforced Reasoning for Embodied Planning
  4. 2026Imagine-then-Plan: Agent Learning from Adaptive Lookahead with World Models
  5. 2025RIG: Synergizing Reasoning and Imagination in End-to-End Generalist Policy
  6. 2025ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning and Online Reinforcement Learning
  7. 2025ProxyThinker: Test-Time Guidance through Small Visual Reasoners
  8. 2026Seeing Isn't Believing: Mitigating Belief Inertia via Active Intervention in Embodied Agents