# System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcuts

arXiv: https://arxiv.org/abs/2505.18962 · HTML: https://mlsanity.com/paper/2505.18962/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 63 of its 78 references that are in the map.
- First posted on arXiv 25 May 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 30 of 63)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies](https://mlsanity.com/paper/2101.02235/) (2021)
- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025)
- [Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs](https://mlsanity.com/paper/2412.21187/) (2024)
- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference](https://mlsanity.com/paper/2004.12993/) (2020)
- [From System 1 to System 2: A Survey of Reasoning Large Language Models](https://mlsanity.com/paper/2502.17419/) (2025)
- [Confident Adaptive Language Modeling](https://mlsanity.com/paper/2207.07061/) (2022)

## Built on by (most cited first, 27 of 27)

- [A Survey on Latent Reasoning](https://mlsanity.com/paper/2507.06203/) (2025)
- [Implicit Reasoning in Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2509.02350/) (2025)
- [The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook](https://mlsanity.com/paper/2604.02029/) (2026)
- [SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMs](https://mlsanity.com/paper/2510.05069/) (2025)
- [Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey](https://mlsanity.com/paper/2507.09662/) (2025)
- [Toward Efficient Agents: Memory, Tool learning, and Planning](https://mlsanity.com/paper/2601.14192/) (2026)
- [KaVa: Latent Reasoning via Compressed KV-Cache Distillation](https://mlsanity.com/paper/2510.02312/) (2025)
- [SeLaR: Selective Latent Reasoning in Large Language Models](https://mlsanity.com/paper/2604.08299/) (2026)
- [Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization](https://mlsanity.com/paper/2601.21358/) (2026)
- [Learning to Ponder: Adaptive Reasoning in Latent Space](https://mlsanity.com/paper/2509.24238/) (2025)
- [SPOT: Span-level Pause-of-Thought for Efficient and Interpretable Latent Reasoning in Large Language Models](https://mlsanity.com/paper/2603.06222/) (2026)
- [LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails](https://mlsanity.com/paper/2605.17329/) (2026)
- [Self-Policy Distillation via Capability-Selective Subspace Projection](https://mlsanity.com/paper/2605.22675/) (2026)
- [Capabilities and Fundamental Limits of Latent Chain-of-Thought](https://mlsanity.com/paper/2602.01148/) (2026)
- [LightThinker++: From Reasoning Compression to Memory Management](https://mlsanity.com/paper/2604.03679/) (2026)
- [Optimizing Reasoning Efficiency through Prompt Difficulty Prediction](https://mlsanity.com/paper/2511.03808/) (2025)
- [Semantic-Enriched Latent Visual Reasoning](https://mlsanity.com/paper/2605.19342/) (2026)
- [Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Intervention](https://mlsanity.com/paper/2606.01243/) (2026)
- [Latent Abstraction for Retrieval-Augmented Generation](https://mlsanity.com/paper/2604.17866/) (2026)
- [Reasoning-aware Speculative Decoding for Efficient Vision-Language-Action Models in Autonomous Driving](https://mlsanity.com/paper/2606.31160/) (2026)
- [Efficient Reasoning on the Edge](https://mlsanity.com/paper/2603.16867/) (2026)
- [State of Thought Enables Endogenous Reasoning](https://mlsanity.com/paper/2609.16055/) (2026)
- [LatentHarness: Learning Latent Actions for Memory and Reasoning via Counterfactual Policy Distillation](https://mlsanity.com/paper/2609.39740/) (2026)
- [MemCodex: Self-Programming Hierarchical Memory for Language Agents](https://mlsanity.com/paper/2609.39765/) (2026)
- [SanSi: A Looped Typed Decision Model for System 1.5 Thinking](https://mlsanity.com/paper/2610.07730/) (2026)
- [Coarse-Graining Recursive Expert Computation: Continuation-Relative Interfaces and Failure Modes](https://mlsanity.com/paper/2610.14413/) (2026)
- [What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds](https://mlsanity.com/paper/2610.20195/) (2026)

## Abstract (arXiv)

Chain-of-thought (CoT) reasoning enables large language models (LLMs) to move beyond fast System-1 responses and engage in deliberative System-2 reasoning. However, this comes at the cost of significant inefficiency due to verbose intermediate output. Recent latent-space reasoning methods improve efficiency by operating on hidden states without decoding into language, yet they treat all steps uniformly, failing to distinguish critical deductions from auxiliary steps and resulting in suboptimal use of computational resources. In this paper, we propose System-1.5 Reasoning, an adaptive reasoning framework that dynamically allocates computation across reasoning steps through shortcut paths in latent space. Specifically, System-1.5 Reasoning introduces two types of dynamic shortcuts. The model depth shortcut (DS) adaptively reasons along the vertical depth by early exiting non-critical tokens through lightweight adapter branches, while allowing critical tokens to continue through deeper Transformer layers. The step shortcut (SS) reuses hidden states across the decoding steps to skip trivial steps and reason horizontally in latent space. Training System-1.5 Reasoning involves a two-stage self-distillation process: first distilling natural language CoT into latent-space continuous thought, and then distilling full-path System-2 latent reasoning into adaptive shortcut paths (System-1.5 Reasoning). Experiments on reasoning tasks demonstrate the superior performance of our method. For example, on GSM8K, System-1.5 Reasoning achieves reasoning performance comparable to traditional CoT fine-tuning methods while accelerating inference by over 20x and reducing token generation by 92.31% on average.