mlsanityData to 7 Oct 2026

Recall with Reasoning: Chain-of-Thought Distillation for Mamba's Long-Context Memory and Extrapolation

arXiv 2505.03320 ↗First posted 6 May 2025cs.CLcited by 2

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Mamba's theoretical infinite-context potential is limited in practice when sequences far exceed training lengths. This work explores unlocking Mamba's long-context memory ability by a simple-yet-effective method, Recall with Reasoning (RwR), by distilling chain-of-thought (CoT) summarization from a teacher model. Specifically, RwR prepends these summarization as CoT prompts during fine-tuning, teaching Mamba to actively recall and reason over long contexts. Experiments on LONGMEMEVAL and HELMET show RwR boosts Mamba's long-context performance against comparable Transformer/hybrid baselines under similar pretraining conditions, while preserving short-context capabilities, all without architectural changes.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 34 references that are in the map. First posted on arXiv 6 May 2025.

Builds on

16 of 34 references.

References · 16

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  6. 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 390
  7. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  8. 2024Long-context LLMs Struggle with Long In-context Learningcited by 285
  9. 2024An Empirical Study of Mamba-based Language Modelscited by 177
  10. 2024Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modelingcited by 167
  11. 2024HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughlycited by 119
  12. 2024Hymba: A Hybrid-head Architecture for Small Language Modelscited by 95
4 more
  1. 2024Falcon Mamba: The First Competitive Attention-free 7B Language Modelcited by 43
  2. 2024Disentangling Memory and Reasoning Ability in Large Language Modelscited by 43
  3. 2024ReMamba: Equip Mamba with Effective Long-Sequence Modelingcited by 12
  4. 2023Distilling Large Language Models using Skill-Occupation Graph Context for HR-Related Taskscited by 3

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Revisiting Test-Time Scaling: A Survey and a Diversity-Aware Method for Efficient Reasoningcited by 8
  2. 2025Merge-of-Thought Distillationcited by 6

Similar, not cited

Close in wording, no citation either way

  1. 2024Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons
  2. 2025Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning
  3. 2025Attention Reveals More Than Tokens: Training-Free Long-Context Reasoning with Attention-guided Retrieval
  4. 2025The Challenge of Teaching Reasoning to LLMs Without RL or Distillation
  5. 2026When Chain-of-Thought Helps and When It Hurts: An Empirical Investigation of the Serial-Depth Bottleneck in LLM Reasoning
  6. 2026Enhanced Mycelium of Thought (EMoT): A Bio-Inspired Hierarchical Reasoning Architecture with Strategic Dormancy and Mnemonic Encoding
  7. 2025Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pre-trained Priors
  8. 2025SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought