mlsanityData to 7 Oct 2026

MeTHanol: Modularized Thinking Language Models with Intermediate Layer Thinking, Decoding and Bootstrapping Reasoning

arXiv 2409.12059 ↗First posted 18 Sep 2024cs.CL cs.AI cs.LGcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Current research efforts are focused on enhancing the thinking and reasoning capability of large language model (LLM) by prompting, data-driven emergence and inference-time computation. In this study, we consider stimulating language model's thinking and cognitive abilities from a modular perspective, which mimics the human brain architecture. We select a specific intermediate attention layer with newly implemented language heads. We conduct dual-layer fine-tuning by annotated (query, thought, answer) samples and show that the intermediate layer can also learn to decode fluent and reasonable language tokens. A two-pass inference mechanism is designed to generate thoughts then formal responses. The entire framework is called modularized thinking language model (MeTHanol) which can enhance LLM's cognitive behaviors as indicated by Theory of Mind (ToM) and Vignette-based experiments. Case studies also show that MeTHanol can plan and self-reflect and generate human-like thoughts and answers, even on unseen and open-domain tasks. MeTHanol can also adapt to a personalized prompt and behave as the specified character. Our study holds promise for significant cognitive gains from a modular perspective. Our code, model and data are available at https://bachozean.github.io/methanol-page

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 6 of its 30 references that are in the map. First posted on arXiv 18 Sep 2024.

Builds on

6 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 5

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2023A Comprehensive Overview of Large Language Modelscited by 639
  3. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  4. 2023Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilitiescited by 86
  5. 2024Self-Reflection in LLM Agents: Effects on Problem-Solving Performancecited by 38

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025On the Thinking-Language Modeling Gap in Large Language Models
  2. 2026Thinking vs. NoThinking: Towards Interpreting Reasoning Mechanisms of Large Language Models via Sparse Autoencoders
  3. 2023Why Can Large Language Models Generate Correct Chain-of-Thoughts?
  4. 2025DynamicMind: A Tri-Mode Thinking System for Large Language Models
  5. 2024Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation
  6. 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models
  7. 2026Understanding Large Language Models
  8. 2023MindGames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal Logic