mlsanityData to 14 Oct 2026

Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning

arXiv 2502.03275 ↗First posted 5 Feb 2025cs.CL cs.AI cs.LGcited by 91

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large Language Models (LLMs) excel at reasoning and planning when trained on chainof-thought (CoT) data, where the step-by-step thought process is explicitly outlined by text tokens. However, this results in lengthy inputs where many words support textual coherence rather than core reasoning information, and processing these inputs consumes substantial computation resources. In this work, we propose a hybrid representation of the reasoning process, where we partially abstract away the initial reasoning steps using latent discrete tokens generated by VQ-VAE, significantly reducing the length of reasoning traces. We explore the use of latent trace abstractions in two scenarios: 1) training the model from scratch for the Keys-Finding Maze problem, 2) fine-tuning LLMs on this hybrid data with an extended vocabulary including unseen latent tokens, for both logical and mathematical reasoning problems. To facilitate effective learning, we introduce a simple training procedure that randomly mixes latent and text tokens, which enables fast adaptation to new latent tokens. Our approach consistently outperforms the baselines methods in various benchmarks.

Key facts. Cited by 91 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 44 of its 51 references that are in the map. First posted on arXiv 5 Feb 2025.

Builds on

44 of 51 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Deliberation in Latent Space via Differentiable Cache Augmentation23 Dec
  2. 2024Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces13 Oct
  3. 2024Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics7 May
  4. 2024Stream of Search (SoS): Learning to Search in Language1 Apr
  5. 2023Think before you speak: Training Language Models With Pause Tokens3 Oct
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May

Other references · 38

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2024Qwen2.5 Technical Reportcited by 4,942
  8. 2017Neural Discrete Representation Learningcited by 4,712
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  11. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  12. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
26 more
  1. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  2. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  3. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  4. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  5. 2023Llemma: An Open Language Model For Mathematicscited by 388
  6. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 337
  7. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 335
  8. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 294
  9. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 260
  10. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Stepcited by 235
  11. 2024Chain-of-Thought Reasoning Without Promptingcited by 210
  12. 2023TheoremQA: A Theorem-driven Question Answering datasetcited by 192
  13. 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Modelscited by 191
  14. 2023Implicit Chain of Thought Reasoning via Knowledge Distillationcited by 176
  15. 2024Distilling System 2 into System 1cited by 159
  16. 2024Byte Latent Transformer: Patches Scale Better Than Tokenscited by 146
  17. 2024MathScale: Scaling Instruction Tuning for Mathematical Reasoningcited by 144
  18. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 108
  19. 2024DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solvingcited by 95
  20. 2024Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrappingcited by 90
  21. 2024Large Concept Models: Language Modeling in a Sentence Representation Spacecited by 89
  22. 2023Guiding Language Model Reasoning with Planning Tokenscited by 58
  23. 2022Efficient Planning in a Compact Latent Action Spacecited by 57
  24. 2023Forward-Backward Reasoning in Large Language Models for Mathematical Verificationcited by 40
  25. 2024From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiencycited by 29
  26. 2023H-GAP: Humanoid Control with a Generalist Plannercited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 45; 2026: 46.

91 papers.

Sort:

By area:

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 464
  2. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 225
  3. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  4. 2025Dynamic Early Exit in Reasoning Modelscited by 219
  5. 2025Reasoning Models Can Be Effective Without Thinkingcited by 187
  6. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 132
  7. 2025Efficient Reasoning Models: A Surveycited by 95
  8. 2025Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chainscited by 85
  9. 2025A Survey on Latent Reasoningcited by 81
  10. 2025Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thoughtcited by 77
  11. 2025SIM-CoT: Supervised Implicit Chain-of-Thoughtcited by 70
  12. 2025Efficient Inference for Large Reasoning Models: A Surveycited by 61
38 more
  1. 2025Inverse Scaling in Test-Time Computecited by 50
  2. 2025Reinforced Latent Reasoning for LLM-based Recommendationcited by 47
  3. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 43
  4. 2025Hybrid Latent Reasoning via Reinforcement Learningcited by 32
  5. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoningcited by 29
  6. 2025System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcutscited by 27
  7. 2025SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMscited by 22
  8. 2025SofT-GRPO: Surpassing Discrete-Token LLM Reinforcement Learning via Gumbel-Reparameterized Soft-Thinking Policy Optimizationcited by 21
  9. 2025HAPO: Training Language Models to Reason Concisely via History-Aware Policy Optimizationcited by 19
  10. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  11. 2025OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking Systemcited by 18
  12. 2025Emergence of Superposition: Unveiling the Training Dynamics of Chain of Continuous Thoughtcited by 16
  13. 2025Think-at-Hard: Dynamic Looped Transformers for Improved Reasoningcited by 16
  14. 2025GaLore 2: Large-Scale LLM Pre-Training by Gradient Low-Rank Projectioncited by 14
  15. 2026Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structurecited by 11
  16. 2026Next Concept Prediction in Discrete Latent Space Leads to Stronger Language Modelscited by 11
  17. 2025ConciseHint: Boosting Efficient Reasoning via Continuous Concise Hints during Generationcited by 10
  18. 2026Residual Context Diffusion Language Modelscited by 10
  19. 2026AdaPonderLM: Gated Pondering Language Models with Token-Wise Adaptive Depthcited by 10
  20. 2026Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanationcited by 10
  21. 2025A*-Thought: Efficient Reasoning via Bidirectional Compression for Low-Resource Settingscited by 9
  22. 2025Can LLMs Reason Abstractly Over Math Word Problems Without CoT? Disentangling Abstract Formulation From Arithmetic Computationcited by 8
  23. 2025Long or short CoT? Investigating Instance-level Switch of Large Reasoning Modelscited by 8
  24. 2025KaVa: Latent Reasoning via Compressed KV-Cache Distillationcited by 8
  25. 2025SemCoT: Accelerating Chain-of-Thought Reasoning through Semantically-Aligned Implicit Tokenscited by 8
  26. 2025LaRe: Latent Refocusing for Multimodal Reasoningcited by 8
  27. 2026SeLaR: Selective Latent Reasoning in Large Language Modelscited by 7
  28. 2026T^2MLR: Transformer with Temporal Middle-Layer Recurrencecited by 7
  29. 2025Context-level Language Modeling by Learning Predictive Context Embeddingscited by 6
  30. 2025PREMISE: Scalable and Strategic Prompt Optimization for Efficient Mathematical Reasoning in Large Modelscited by 5
  31. 2026Thinking in Latents: Adaptive Anchor Refinement for Implicit Reasoning in LLMscited by 5
  32. 2026Not All LLM Reasoning is Visible in the Chain-of-Thoughtcited by 5
  33. 2025Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanismcited by 4
  34. 2025Efficient Turing Machine Simulation with Transformerscited by 4
  35. 2026Capabilities and Fundamental Limits of Latent Chain-of-Thoughtcited by 4
  36. 2026Internalizing LLM Reasoning via Discovery and Replay of Latent Actionscited by 4
  37. 2026The Art of Efficient Reasoning: Data, Reward, and Optimizationcited by 4
  38. 2026Efficient Reasoning with Balanced Thinkingcited by 4

50 most cited of 91 All 91 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Extending Token Computation for LLM Reasoning
  2. 2025Do Latent Tokens Think? A Causal and Adversarial Analysis of Chain-of-Continuous-Thought
  3. 2026CoLT: Reasoning with Chain of Latent Tool Calls
  4. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
  5. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  6. 2025Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models
  7. 2024Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding
  8. 2026Tyler: Typed Latent Reasoning for Language Models -- When to Think, What to Compute, and How Much to Allocate