mlsanityData to 7 Oct 2026

SuperThoughts: Reasoning Tokens in Superposition

arXiv 2606.13862 ↗First posted 11 Jun 2026cs.LG cs.AI cs.CLcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Long Chain-of-Thought (CoT) reasoning improves LLM problem-solving but is computationally expensive due to sequential token generation. While recent works explore reasoning in continuous latent spaces to bypass discrete token generation, they often struggle with training stability and fail to scale to complex, long-horizon tasks due to lack of supervision signal. We propose SuperThoughts, which compresses pairs of consecutive CoT tokens into single latent representations and decodes two tokens per step via a lightweight Multi-Token Prediction (MTP) module. This preserves discrete token supervision at training time while doubling throughput at inference time. We finetune Qwen2.5-Math-1.5B-Instruct, Qwen2.5-Math-7B-Instruct, Qwen2.5-Math-14B-Instruct, and evaluate on MATH500, AMC, OlympiadBench, and GPQA-Diamond. With a confidence-based adaptive mechanism that falls back to standard decoding when uncertain, SuperThoughts achieves ~20--30% CoT length reduction while maintaining accuracy with minimal degradation (1-2 points accuracy drop on most tasks).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 27 of its 53 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

27 of 53 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Dynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Space31 Dec
  2. 2025LLM Latent Reasoning as Chain of Superposition17 Oct
  3. 2025Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains22 May
  4. 2024Compressed Chain of Thought: Efficient Reasoning Through Dense Representations17 Dec
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug

Other references · 22

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2024OpenAI o1 System Cardcited by 1,220
  4. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  5. 2024DeepSeek-V3 Technical Reportcited by 724
  6. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 287
  7. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  8. 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning datasetcited by 165
  9. 2025MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttrainingcited by 117
  10. 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 109
  11. 2025Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Modelscited by 104
  12. 2025Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoningcited by 75
10 more
  1. 2025Dynamic Chunking for End-to-End Hierarchical Sequence Modelingcited by 65
  2. 2025LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinkingcited by 34
  3. 2025Text Generation Beyond Discrete Token Samplingcited by 21
  4. 2026Multiplex Thinking: Reasoning via Token-wise Branch-and-Mergecited by 16
  5. 2025Efficient Joint Prediction of Multiple Future Tokenscited by 10
  6. 2025No Mean Feat: Simple, Strong Baselines for Context Compressioncited by 6
  7. 2025CompLLM: Compression for Long Context Q&Acited by 5
  8. 2025HAMburger: Accelerating LLM Inference via Token Smashingcited by 4
  9. 2025Learning to Reason with Mixture of Tokenscited by 3
  10. 2026Efficient Pre-Training with Token Superpositioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Efficient Reasoning via Constrained Optimization in Latent Space
  2. 2026Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens
  3. 2026Latent Reasoning with Supervised Thinking States
  4. 2025VeriThinker: Learning to Verify Makes Reasoning Model Efficient
  5. 2025Soft Tokens, Hard Truths
  6. 2025Not All Tokens Are What You Need In Thinking
  7. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMs
  8. 2025Pruning the Unsurprising: Efficient LLM Reasoning via First-Token Surprisal