mlsanityData to 14 Oct 2026

Intern-S2-Mobius: Foundation Model with Decoupled Knowledge and Reasoning

arXiv 2608.14290 ↗First posted 14 Aug 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce Mobius-v0, an architecture that comprises a globally shared Memory (FFN) that stores knowledge vectors and multiple Reasoners (Self-Attn) that iteratively achieve compositional reasoning. Using hidden states as cache and carrier, reasoners repeatedly query memory for required knowledge-vectors, while the knowledge is transmitted back to reasoning operators. Through this knowledge-reasoning-separation architecture, Mobius achieves better knowledge compression and reasoning efficiency. Built upon Mobius-v0 architecture: 1) Our 7B model trained-from-scratch achieves similar downstream score as a 7B Transformer baseline with 62.6% of baseline's training data. 2) Our Intern-S2-Mobius, continually-pretrained from Qwen3.5-35B, achieves similar downstream score while delivering nearly 4x end-to-end inference speedup.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 65 of its 75 references that are in the map. First posted on arXiv 14 Aug 2026.

Builds on

65 of 75 references.

References · 65

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  8. 2016Densely Connected Convolutional Networkscited by 6,720
  9. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  10. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  11. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  12. 2022Training Compute-Optimal Large Language Modelscited by 3,019
38 more
  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  3. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  4. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  5. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  6. 2020Rethinking Attention with Performerscited by 1,601
  7. 2018World Modelscited by 1,513
  8. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  9. 2025s1: Simple test-time scalingcited by 1,341
  10. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  11. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  12. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  13. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  14. 2023Mastering Diverse Domains through World Modelscited by 1,253
  15. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  16. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  17. 2020On Layer Normalization in the Transformer Architecturecited by 922
  18. 2025Large Language Diffusion Modelscited by 912
  19. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  20. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  21. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 578
  22. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
  23. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 415
  24. 2025Accelerating scientific discovery with Co-Scientistcited by 401
  25. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 365
  26. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  27. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 323
  28. 2023Think before you speak: Training Language Models With Pause Tokenscited by 264
  29. 2015Highway Networkscited by 260
  30. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMscited by 249
  31. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 240
  32. 2023Looped Transformers as Programmable Computerscited by 237
  33. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Stepcited by 235
  34. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 225
  35. 2024Simple linear attention language models balance the recall-throughput tradeoffcited by 208
  36. 2024Compressed Chain of Thought: Efficient Reasoning Through Dense Representationscited by 201
  37. 2025Reasoning Models Can Be Effective Without Thinkingcited by 187
  38. 2023Implicit Chain of Thought Reasoning via Knowledge Distillationcited by 176

50 most cited of 65 All 65 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Reuse, Don't Recompute: Efficient Large Reasoning Model Inference via Memory Orchestration
  2. 2025Apriel-H1: Towards Efficient Enterprise Reasoning Models
  3. 2025Hydra: A Modular Architecture for Efficient Long-Context Reasoning
  4. 2026ReasonAlloc: Hierarchical Decoding-Time KV Cache Budget Allocation for Reasoning Models
  5. 2026MEMENTO: Teaching LLMs to Manage Their Own Context
  6. 2025Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task Arithmetic
  7. 2026Efficient Reasoning on the Edge
  8. 2025Improving Reasoning Performance in Large Language Models via Representation Engineering