mlsanityData to 14 Oct 2026

DSADF: Thinking Fast and Slow for Decision Making

arXiv 2505.08189 ↗First posted 13 May 2025cs.LG cs.AIcited by 7

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Although Reinforcement Learning (RL) agents are effective in well-defined environments, they often struggle to generalize their learned policies to dynamic settings due to their reliance on trial-and-error interactions. Recent work has explored applying Large Language Models (LLMs) or Vision Language Models (VLMs) to boost the generalization of RL agents through policy optimization guidance or prior knowledge. However, these approaches often lack seamless coordination between the RL agent and the foundation model, leading to unreasonable decision-making in unfamiliar environments and efficiency bottlenecks. Making full use of the inferential capabilities of foundation models and the rapid response capabilities of RL agents and enhancing the interaction between the two to form a dual system is still a lingering scientific question. To address this problem, we draw inspiration from Kahneman's theory of fast thinking (System 1) and slow thinking (System 2), demonstrating that balancing intuition and deep reasoning can achieve nimble decision-making in a complex world. In this study, we propose a Dual-System Adaptive Decision Framework (DSADF), integrating two complementary modules: System 1, comprising an RL agent and a memory space for fast and intuitive decision making, and System 2, driven by a VLM for deep and analytical reasoning. DSADF facilitates efficient and adaptive decision-making by combining the strengths of both systems. The empirical study in the video game environment: Crafter and Housekeep demonstrates the effectiveness of our proposed method, showing significant improvements in decision abilities for both unseen and known tasks.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 72 of its 84 references that are in the map. First posted on arXiv 13 May 2025.

Builds on

72 of 84 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Let's Verify Step by Step31 May

Other references · 71

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  8. 2024Qwen2.5 Technical Reportcited by 4,942
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  10. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  11. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  12. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
38 more
  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  2. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  3. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  4. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  5. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  6. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  8. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  9. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  10. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
  11. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  12. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  13. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
  14. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  15. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 316
  16. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  17. 2022Reasoning with Language Model Prompting: A Surveycited by 252
  18. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 221
  19. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  20. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
  21. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 195
  22. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 187
  23. 2019A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptationcited by 177
  24. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  25. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 174
  26. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  27. 2018Self-Imitation Learningcited by 138
  28. 2024Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphscited by 119
  29. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 116
  30. 2019Model-free Deep Reinforcement Learning for Urban Autonomous Drivingcited by 98
  31. 2021Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformerscited by 93
  32. 2023Large Language Models as Generalizable Policies for Embodied Taskscited by 86
  33. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Modelscited by 70
  34. 2023Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraftcited by 61
  35. 2023A Survey of Temporal Credit Assignment in Deep Reinforcement Learningcited by 59
  36. 2022Housekeep: Tidying Virtual Households using Commonsense Reasoningcited by 57
  37. 2024Visual Agents as Fast and Slow Thinkerscited by 53
  38. 2023Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Modelscited by 52

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025Plan Then Action:High-Level Planning Guidance Reinforcement Learning for LLM Reasoningcited by 19
  2. 2025Beyond Empathy: Integrating Diagnostic and Therapeutic Reasoning with Large Language Models for Mental Health Counselingcited by 18
  3. 2025Learning Deliberately, Acting Intuitively: Unlocking Test-Time Reasoning in Multimodal LLMscited by 1
  4. 2026Think2: Grounded Metacognitive Reasoning in Large Language Modelscited by 1
  5. 2026ReShift: Aha-Moment-Driven Reasoning-Level Backdoor Attacks on Vision-Language Modelscited by 0
  6. 2026FedCausal-Dyn: A Causal-Dynamic Paradigm for Federated Learning under Dynamic Feature Driftcited by 0
  7. 2026Graph-Based Modeling of Financial Volatility Dynamicscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Rule-Bottleneck Reinforcement Learning: Joint Explanation and Decision Optimization for Resource Allocation with Language Agents
  2. 2026All Roads Lead to Rome: Incentivizing Divergent Thinking in Vision-Language Models
  3. 2025The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learning
  4. 2025Fast-Slow Thinking GRPO for Large Vision-Language Model Reasoning
  5. 2024A Survey On Enhancing Reinforcement Learning in Complex Environments: Insights from Human and LLM Feedback
  6. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
  7. 2025A Survey of Reinforcement Learning for Large Reasoning Models
  8. 2025A Technical Survey of Reinforcement Learning Techniques for Large Language Models