mlsanityData to 14 Oct 2026

On the Thinking-Language Modeling Gap in Large Language Models

arXiv 2505.12896 ↗First posted 19 May 2025cs.CL cs.LG stat.MLcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

System 2 reasoning is one of the defining characteristics of intelligence, which requires slow and logical thinking. Human conducts System 2 reasoning via the language of thoughts that organizes the reasoning process as a causal sequence of mental language, or thoughts. Recently, it has been observed that System 2 reasoning can be elicited from Large Language Models (LLMs) pre-trained on large-scale natural languages. However, in this work, we show that there is a significant gap between the modeling of languages and thoughts. As language is primarily a tool for humans to share knowledge and thinking, modeling human language can easily absorb language biases into LLMs deviated from the chain of thoughts in minds. Furthermore, we show that the biases will mislead the eliciting of "thoughts" in LLMs to focus only on a biased part of the premise. To this end, we propose a new prompt technique termed Language-of-Thoughts (LoT) to demonstrate and alleviate this gap. Instead of directly eliciting the chain of thoughts from partial information, LoT instructs LLMs to adjust the order and token used for the expressions of all the relevant information. We show that the simple strategy significantly reduces the language modeling biases in LLMs and improves the performance of LLMs across a variety of reasoning tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 30 of its 74 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

30 of 74 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024DeepSeek-V3 Technical Report27 Dec
  2. 2024"Let's Argue Both Sides": Argument Generation Can Force Small Models to Utilize Previously Inaccessible Reasoning Capabilities16 Oct
  3. 2024Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities4 Jun
  4. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking14 Mar
  5. 2023LLaMA: Open and Efficient Foundation Language Models27 Feb
  6. 2022FOLIO: Natural Language Reasoning with First-Order Logic2 Sep

Other references · 24

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  3. 2024Qwen2 Technical Reportcited by 2,120
  4. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  6. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  7. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 659
  8. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  9. 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 270
  10. 2024Chain-of-Thought Reasoning Without Promptingcited by 210
  11. 2021Towards Causal Representation Learningcited by 209
  12. 2024The pitfalls of next-token predictioncited by 160
12 more
  1. 2024Distilling System 2 into System 1cited by 159
  2. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  3. 2017The Consciousness Priorcited by 107
  4. 2024Chain of Thoughtlessness? An Analysis of CoT in Planningcited by 103
  5. 2023System 2 Attention (is something you might need too)cited by 89
  6. 2024Premise Order Matters in Reasoning with Large Language Modelscited by 72
  7. 2024Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Modelscited by 68
  8. 2023Contrastive Chain-of-Thought Promptingcited by 52
  9. 2023The Expressive Power of Transformers with Chain of Thoughtcited by 50
  10. 2024Deciphering the Factors Influencing the Efficacy of Chain-of-Thought: Probability, Memorization, and Noisy Reasoningcited by 27
  11. 2024Prompting Fairness: Integrating Causality to Debias Large Language Modelscited by 21
  12. 2024Strong hallucinations from negation and how to fix themcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Why Can Large Language Models Generate Correct Chain-of-Thoughts?
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Models
  3. 2025Language and Thought: The View from LLMs
  4. 2023Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilities
  5. 2023Understanding Social Reasoning in Language Models with Language Models
  6. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought
  7. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  8. 2025Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models