Path-Lock Expert: Separating Reasoning Mode in Hybrid Thinking via Architecture-Level Separation
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Hybrid-thinking language models expose explicit /think and /no_think modes, but current designs do not separate them cleanly. Even in /no_think mode, models often emit long and self-reflective responses, causing reasoning leakage. Existing work reduces this issue through better data curation and multi-stage training, yet leakage remains because both modes are still encoded in the same feed-forward parameters. We propose Path-Lock Expert (PLE), an architecture-level solution that replaces the single MLP in each decoder layer with two semantically locked experts, one for /think and one for /no_think, while keeping attention, embeddings, normalization, and the language-model head shared. A deterministic control-token router selects exactly one expert path for the entire sequence, so inference preserves the dense model's per-token computation pattern and each expert receives mode-pure updates during supervised fine-tuning. Across math and science reasoning benchmarks, PLE maintains strong /think performance while producing a substantially stronger mode separation, a /no_think mode with higher accuracy and far less reasoning leakage. On Qwen3-4B, for example, compared to the SFT-only baseline on AIME24, PLE generates 17x fewer reflective tokens (6.01 vs. 0.35 per answer) and 2x shorter outputs (8665 vs. 4101 tokens), and improves /no_think accuracy from 35.33% to 44.67%, while maintaining /think-mode performance (61.33% vs. 60.00%). These results suggest that controllable hybrid thinking is fundamentally an architectural problem, and separating mode-specific feed-forward pathways is a simple and effective solution.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 59 of its 78 references that are in the map. First posted on arXiv 29 Apr 2026.
Builds on
59 of 78 references.
Closest ancestors · 11
Same direction, newest first
- 2026Mid-Think: Training-Free Intermediate-Budget Reasoning via Token-Level Triggers11 Jan
- 2025Demystifying Hybrid Thinking: Can LLMs Truly Switch Between Think and No-Think?14 Oct
- 2025gpt-oss-120b & gpt-oss-20b Model Card8 Aug
- 2025Grammars of Formal Uncertainty: When to Trust LLMs in Automated Reasoning Tasks26 May
- 2025Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Time12 Apr
- 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models20 Mar
- 2025The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Models4 Mar
- 2025LightThinker: Thinking Step-by-Step Compression21 Feb
- 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMs17 Feb
- 2025s1: Simple test-time scaling31 Jan
- 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
Other references · 48
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2025Qwen3 Technical Reportcited by 8,195
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2023Let's Verify Step by Stepcited by 3,598
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
36 more
- 2024Mixtral of Expertscited by 1,785
- 2025Gemma 3 Technical Reportcited by 1,686
- 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2020Gradient Surgery for Multi-Task Learningcited by 1,210
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
- 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
- 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
- 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
- 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
- 2021Knowledge Neurons in Pretrained Transformerscited by 590
- 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
- 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
- 2025LIMO: Less is More for Reasoningcited by 488
- 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
- 2025Chain of Draft: Thinking Faster by Writing Lesscited by 290
- 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!cited by 132
- 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 116
- 202570% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float (DFloat11)cited by 23
- 2024Proof of Thought : Neurosymbolic Program Synthesis allows Robust and Interpretable Reasoningcited by 17
- 2026Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoningcited by 13
- 2025FAEDKV: Infinite-Window Fourier Transform for Unbiased KV Cache Compressioncited by 11
- 2026VERGE: Formal Refinement and Guidance Engine for Verifiable LLM Reasoningcited by 11
- 2024Forte : Finding Outliers with Representation Typicality Estimationcited by 9
- 2025Quantize What Counts: More for Keys, Less for Valuescited by 9
- 2025Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMscited by 8
- 2025K4: Online Log Anomaly Detection Via Unsupervised Typicality Learningcited by 6
- 2025LABELING COPILOT: A Deep Research Agent for Automated Data Curation in Computer Visioncited by 6
- 2026When Domains Interact: Asymmetric and Order-Sensitive Cross-Domain Effects in Reinforcement Learning for Reasoningcited by 5
- 2024Visual Concept Networks: A Graph-Based Approach to Detecting Anomalous Data in Deep Neural Networkscited by 4
- 2025Efficient Fine-Grained GPU Performance Modeling for Distributed Deep Learning of LLMcited by 4
- 2026Toward Guarantees for Clinical Reasoning in Vision Language Models via Formal Verificationcited by 4
- 2026Trust The Typicalcited by 3
- 2026AgentCE-Bench: Agent Configurable Evaluation with Scalable Horizons and Controllable Difficulty under Lightweight Environmentscited by 3
- 2025Metis-HOME: Hybrid Optimized Mixture-of-Experts for Multimodal Reasoningcited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026HRBench: Benchmarking and Understanding Thinking-Mode Switch Strategies in Hybrid-Reasoning LLMs
- 2026HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Steps
- 2025ParaThinker: Native Parallel Thinking as a New Paradigm to Scale LLM Test-time Compute
- 2025OThink-R1: Intrinsic Fast/Slow Thinking Mode Switching for Over-Reasoning Mitigation
- 2025Self-Route: Automatic Mode Switching via Capability Estimation for Efficient Reasoning
- 2025ThinkDial: An Open Recipe for Controlling Reasoning Effort in Large Language Models
- 2026ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spaces
- 2026Think Through a Bottleneck: Hourglass Reasoning for Rigorous Induction