mlsanityData to 7 Oct 2026

SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs

arXiv 2502.12134 ↗First posted 17 Feb 2025cs.CLcited by 110

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-Thought (CoT) reasoning enables Large Language Models (LLMs) to solve complex reasoning tasks by generating intermediate reasoning steps. However, most existing approaches focus on hard token decoding, which constrains reasoning within the discrete vocabulary space and may not always be optimal. While recent efforts explore continuous-space reasoning, they often require full-model fine-tuning and suffer from catastrophic forgetting, limiting their applicability to state-of-the-art LLMs that already perform well in zero-shot settings with a proper instruction. To address this challenge, we propose a novel approach for continuous-space reasoning that does not require modifying the LLM. Specifically, we employ a lightweight fixed assistant model to speculatively generate instance-specific soft thought tokens as the initial chain of thoughts, which are then mapped into the LLM's representation space via a trainable projection module. Experimental results on five reasoning benchmarks demonstrate that our method enhances LLM reasoning performance through supervised, parameter-efficient fine-tuning. Source code is available at https://github.com/xuyige/SoftCoT.

Key facts. Cited by 110 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 25 of its 36 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

25 of 36 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Training Large Language Models to Reason in a Continuous Latent Space9 Dec
  2. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning18 Sep
  3. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future27 Sep
  4. 2022Large Language Models are Zero-Shot Reasoners24 May
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar

Other references · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  7. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  8. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  9. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  10. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  11. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  12. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
8 more
  1. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  2. 2023Think before you speak: Training Language Models With Pause Tokenscited by 254
  3. 2024Compressed Chain of Thought: Efficient Reasoning Through Dense Representationscited by 192
  4. 2023Guiding Large Language Models via Directional Stimulus Promptingcited by 94
  5. 2025Efficient Reasoning with Hidden Thinkingcited by 72
  6. 2024On the Impact of Fine-Tuning on Chain-of-Thought Reasoningcited by 32
  7. 2023Efficient Cross-Task Prompt Tuning for Few-Shot Conversational Emotion Recognitioncited by 5
  8. 2024Towards Goal-oriented Prompt Engineering for Large Language Models: A Surveycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 48; 2026: 62.

110 papers.

Sort:

By area:

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  2. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  3. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  4. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 216
  5. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  6. 2025Reasoning Models Can Be Effective Without Thinkingcited by 182
  7. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  8. 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 109
  9. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  10. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 99
  11. 2025Efficient Reasoning Models: A Surveycited by 93
  12. 2025Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chainscited by 81
38 more
  1. 2025SIM-CoT: Supervised Implicit Chain-of-Thoughtcited by 63
  2. 2025Efficient Inference for Large Reasoning Models: A Surveycited by 58
  3. 2025Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendationcited by 50
  4. 2025Think or Not Think: A Study of Explicit Thinking in Rule-Based Visual Reinforcement Fine-Tuningcited by 44
  5. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 42
  6. 2025SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoningcited by 38
  7. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  8. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  9. 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Modelscited by 31
  10. 2025When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoningcited by 26
  11. 2025System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcutscited by 23
  12. 2025Parallel Continuous Chain-of-Thought with Jacobi Iterationcited by 21
  13. 2025ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Modelscited by 21
  14. 2025SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMscited by 20
  15. 2026Fast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent Planningcited by 20
  16. 2025KTAE: A Model-Free Algorithm to Key-Tokens Advantage Estimation in Mathematical Reasoningcited by 19
  17. 2026Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent Reasoningcited by 19
  18. 2026Latent Reasoning VLA: Latent Thinking and Prediction for Vision-Language-Action Modelscited by 19
  19. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  20. 2025OThink-R1: Intrinsic Fast/Slow Thinking Mode Switching for Over-Reasoning Mitigationcited by 17
  21. 2025LazyEviction: Lagged KV Eviction with Attention Pattern Observation for Efficient Long Reasoningcited by 17
  22. 2026Forest Before Trees: Latent Superposition for Efficient Visual Reasoningcited by 16
  23. 2026Multiplex Thinking: Reasoning via Token-wise Branch-and-Mergecited by 16
  24. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  25. 2025SynAdapt: Learning Adaptive Reasoning in Large Language Models via Synthetic Continuous Chain-of-Thoughtcited by 12
  26. 2025Slim-SC: Thought Pruning for Efficient Scaling with Self-Consistencycited by 12
  27. 2026LatentMem: Customizing Latent Memory for Multi-Agent Systemscited by 12
  28. 2026MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolutioncited by 12
  29. 2026Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoningcited by 12
  30. 2025AALC: Large Language Model Efficient Reasoning via Adaptive Accuracy-Length Controlcited by 11
  31. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 11
  32. 2026NextMem: Towards Latent Factual Memory for LLM-based Agentscited by 10
  33. 2026Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structurecited by 9
  34. 2025LatentEvolve: Self-Evolving Test-Time Scaling in Latent Spacecited by 8
  35. 2026FoE: Forest of Errors Makes the First Solution the Best in Large Reasoning Modelscited by 8
  36. 2025SemCoT: Accelerating Chain-of-Thought Reasoning through Semantically-Aligned Implicit Tokenscited by 7
  37. 2025LaRe: Latent Refocusing for Multimodal Reasoningcited by 7
  38. 2026FlashMem: Distilling Intrinsic Latent Memory via Computation Reusecited by 7

50 most cited of 110 All 110 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMs
  2. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
  3. 2026D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models
  4. 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
  5. 2024Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
  6. 2025Eliciting Chain-of-Thought in Base LLMs via Gradient-Based Representation Optimization
  7. 2026Hierarchical Chain-of-Thought: Enhancing LLM Reasoning Performance and Efficiency
  8. 2025Early Stopping Chain-of-thoughts in Large Language Models