ToW: Thoughts of Words Improve Reasoning in Large Language Models
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
We introduce thoughts of words (ToW), a novel training-time data-augmentation method for next-word prediction. ToW views next-word prediction as a core reasoning task and injects fine-grained thoughts explaining what the next word should be and how it is related to the previous contexts in pre-training texts. Our formulation addresses two fundamental drawbacks of existing next-word prediction learning schemes: they induce factual hallucination and are inefficient for models to learn the implicit reasoning processes in raw texts. While there are many ways to acquire such thoughts of words, we explore the first step of acquiring ToW annotations through distilling from larger models. After continual pre-training with only 70K ToW annotations, we effectively improve models' reasoning performances by 7% to 9% on average and reduce model hallucination by up to 10%. At the same time, ToW is entirely agnostic to tasks and applications, introducing no additional biases on labels or semantics.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 20 of its 51 references that are in the map. First posted on arXiv 21 Oct 2024.
Builds on
20 of 51 references.
Closest ancestors · 1
Same direction, newest first
Other references · 19
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
- 2023Mistral 7Bcited by 3,164
- 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
- 2023The Falcon Series of Open Language Modelscited by 535
- 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 368
- 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
- 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Stepcited by 235
- 2024Chain-of-Thought Reasoning Without Promptingcited by 210
- 2023Simple synthetic data reduces sycophancy in large language modelscited by 187
7 more
- 2023Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Modelscited by 148
- 2024Rho-1: Not All Tokens Are What You Needcited by 133
- 2023PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuningcited by 29
- 2024West-of-N: Synthetic Preferences for Self-Improving Reward Modelscited by 28
- 2024Conceptual and Unbiased Reasoning in Language Modelscited by 19
- 2024FamiCom: Further Demystifying Prompts for Language Models with Task-Agnostic Performance Estimationcited by 8
- 2024RATIONALYST: Mining Implicit Rationales for Process Supervision of Reasoningcited by 6
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025BOW: Training Language Models to Reason Over Plausible Next Words
- 2026Latent Recurrent Thoughts: Recurrent Refinement of Proposed Latents for Reasoning with Frozen LLMs
- 2024Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models
- 2025Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models
- 2025Reasoning to Learn from Latent Thoughts
- 2024Think Thrice Before You Act: Progressive Thought Refinement in Large Language Models
- 2025PonderLM-2: Pretraining LLM with Latent Thoughts in Continuous Space
- 2024Rational Metareasoning for Large Language Models