mlsanityData to 7 Oct 2026

Path-Lock Expert: Separating Reasoning Mode in Hybrid Thinking via Architecture-Level Separation

arXiv 2604.27201 ↗First posted 29 Apr 2026cs.CL cs.AI cs.LGcited by 3

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Hybrid-thinking language models expose explicit /think and /no_think modes, but current designs do not separate them cleanly. Even in /no_think mode, models often emit long and self-reflective responses, causing reasoning leakage. Existing work reduces this issue through better data curation and multi-stage training, yet leakage remains because both modes are still encoded in the same feed-forward parameters. We propose Path-Lock Expert (PLE), an architecture-level solution that replaces the single MLP in each decoder layer with two semantically locked experts, one for /think and one for /no_think, while keeping attention, embeddings, normalization, and the language-model head shared. A deterministic control-token router selects exactly one expert path for the entire sequence, so inference preserves the dense model's per-token computation pattern and each expert receives mode-pure updates during supervised fine-tuning. Across math and science reasoning benchmarks, PLE maintains strong /think performance while producing a substantially stronger mode separation, a /no_think mode with higher accuracy and far less reasoning leakage. On Qwen3-4B, for example, compared to the SFT-only baseline on AIME24, PLE generates 17x fewer reflective tokens (6.01 vs. 0.35 per answer) and 2x shorter outputs (8665 vs. 4101 tokens), and improves /no_think accuracy from 35.33% to 44.67%, while maintaining /think-mode performance (61.33% vs. 60.00%). These results suggest that controllable hybrid thinking is fundamentally an architectural problem, and separating mode-specific feed-forward pathways is a simple and effective solution.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 59 of its 78 references that are in the map. First posted on arXiv 29 Apr 2026.

Builds on

59 of 78 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026Mid-Think: Training-Free Intermediate-Budget Reasoning via Token-Level Triggers11 Jan
  2. 2025Demystifying Hybrid Thinking: Can LLMs Truly Switch Between Think and No-Think?14 Oct
  3. 2025gpt-oss-120b & gpt-oss-20b Model Card8 Aug
  4. 2025Grammars of Formal Uncertainty: When to Trust LLMs in Automated Reasoning Tasks26 May
  5. 2025Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Time12 Apr
  6. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models20 Mar
  7. 2025The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Models4 Mar
  8. 2025LightThinker: Thinking Step-by-Step Compression21 Feb
  9. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMs17 Feb
  10. 2025s1: Simple test-time scaling31 Jan
  11. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec

Other references · 48

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  7. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  8. 2023Let's Verify Step by Stepcited by 3,598
  9. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  10. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  11. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  12. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
36 more
  1. 2024Mixtral of Expertscited by 1,785
  2. 2025Gemma 3 Technical Reportcited by 1,686
  3. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  4. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  5. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  6. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  7. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  8. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  9. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  10. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  11. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  12. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  13. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  14. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
  15. 2025LIMO: Less is More for Reasoningcited by 488
  16. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  17. 2025Chain of Draft: Thinking Faster by Writing Lesscited by 290
  18. 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!cited by 132
  19. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 116
  20. 202570% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float (DFloat11)cited by 23
  21. 2024Proof of Thought : Neurosymbolic Program Synthesis allows Robust and Interpretable Reasoningcited by 17
  22. 2026Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoningcited by 13
  23. 2025FAEDKV: Infinite-Window Fourier Transform for Unbiased KV Cache Compressioncited by 11
  24. 2026VERGE: Formal Refinement and Guidance Engine for Verifiable LLM Reasoningcited by 11
  25. 2024Forte : Finding Outliers with Representation Typicality Estimationcited by 9
  26. 2025Quantize What Counts: More for Keys, Less for Valuescited by 9
  27. 2025Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMscited by 8
  28. 2025K4: Online Log Anomaly Detection Via Unsupervised Typicality Learningcited by 6
  29. 2025LABELING COPILOT: A Deep Research Agent for Automated Data Curation in Computer Visioncited by 6
  30. 2026When Domains Interact: Asymmetric and Order-Sensitive Cross-Domain Effects in Reinforcement Learning for Reasoningcited by 5
  31. 2024Visual Concept Networks: A Graph-Based Approach to Detecting Anomalous Data in Deep Neural Networkscited by 4
  32. 2025Efficient Fine-Grained GPU Performance Modeling for Distributed Deep Learning of LLMcited by 4
  33. 2026Toward Guarantees for Clinical Reasoning in Vision Language Models via Formal Verificationcited by 4
  34. 2026Trust The Typicalcited by 3
  35. 2026AgentCE-Bench: Agent Configurable Evaluation with Scalable Horizons and Controllable Difficulty under Lightweight Environmentscited by 3
  36. 2025Metis-HOME: Hybrid Optimized Mixture-of-Experts for Multimodal Reasoningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026HRBench: Benchmarking and Understanding Thinking-Mode Switch Strategies in Hybrid-Reasoning LLMs
  2. 2026HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Steps
  3. 2025ParaThinker: Native Parallel Thinking as a New Paradigm to Scale LLM Test-time Compute
  4. 2025OThink-R1: Intrinsic Fast/Slow Thinking Mode Switching for Over-Reasoning Mitigation
  5. 2025Self-Route: Automatic Mode Switching via Capability Estimation for Efficient Reasoning
  6. 2025ThinkDial: An Open Recipe for Controlling Reasoning Effort in Large Language Models
  7. 2026ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spaces
  8. 2026Think Through a Bottleneck: Hourglass Reasoning for Rigorous Induction