# MoM: Mixtures of Scenario-Aware Document Memories for Retrieval-Augmented Generation Systems

arXiv: https://arxiv.org/abs/2510.14252 · HTML: https://mlsanity.com/paper/2510.14252/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 23 of its 30 references that are in the map.
- First posted on arXiv 16 Oct 2025.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)
- Retrieval-augmented generation / [Multimodal RAG](https://mlsanity.com/rag/multimodal-rag/)

## Builds on (most cited first, 23 of 23)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025)
- [Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG](https://mlsanity.com/paper/2501.09136/) (2025)
- [Rethinking with Retrieval: Faithful Large Language Model Inference](https://mlsanity.com/paper/2301.00303/) (2022)
- [MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation](https://mlsanity.com/paper/2409.05591/) (2024)
- [Dense X Retrieval: What Retrieval Granularity Should We Use?](https://mlsanity.com/paper/2312.06648/) (2023)
- [DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language Models](https://mlsanity.com/paper/2403.10081/) (2024)
- [Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Tasks](https://mlsanity.com/paper/2305.05862/) (2023)
- [CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models](https://mlsanity.com/paper/2401.17043/) (2024)
- [Internal Consistency and Self-Feedback in Large Language Models: A Survey](https://mlsanity.com/paper/2407.14507/) (2024)
- [Don't Forget to Connect! Improving RAG with Graph-based Reranking](https://mlsanity.com/paper/2405.18414/) (2024)
- [Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Models](https://mlsanity.com/paper/2407.04121/) (2024)
- [Is Semantic Chunking Worth the Computational Cost?](https://mlsanity.com/paper/2410.13070/) (2024)
- [LumberChunker: Long-Form Narrative Document Segmentation](https://mlsanity.com/paper/2406.17526/) (2024)
- [A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions](https://mlsanity.com/paper/2507.18910/) (2025)
- [OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain](https://mlsanity.com/paper/2412.13018/) (2024)
- [MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation System](https://mlsanity.com/paper/2503.09600/) (2025)
- [BERM: Training the Balanced and Extractable Representation for Matching to Improve Generalization Ability of Dense Retrieval](https://mlsanity.com/paper/2305.11052/) (2023)
- [CCI3.0-HQ: a large-scale Chinese dataset of high quality designed for pre-training large language models](https://mlsanity.com/paper/2410.18505/) (2024)
- [Humanlike Cognitive Patterns as Emergent Phenomena in Large Language Models](https://mlsanity.com/paper/2412.15501/) (2024)

## Built on by (most cited first, 1 of 1)

- [A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents](https://mlsanity.com/paper/2602.06052/) (2026)

## Abstract (arXiv)

The traditional RAG paradigm, which typically engages in the comprehension of relevant text chunks in response to received queries, inherently restricts both the depth of knowledge internalization and reasoning capabilities. To address this limitation, our research transforms the text processing in RAG from passive chunking to proactive understanding, defining this process as document memory extraction with the objective of simulating human cognitive processes during reading. Building upon this, we propose the Mixtures of scenario-aware document Memories (MoM) framework, engineered to efficiently handle documents from multiple domains and train small language models (SLMs) to acquire the ability to proactively explore and construct document memories. The MoM initially instructs large language models (LLMs) to simulate domain experts in generating document logical outlines, thereby directing structured chunking and core content extraction. It employs a multi-path sampling and multi-perspective evaluation mechanism, specifically designing comprehensive metrics that represent chunk clarity and extraction completeness to select the optimal document memories. Additionally, to infuse deeper human-like reading abilities during the training of SLMs, we incorporate a reverse reasoning strategy, which deduces refined expert thinking paths from high-quality outcomes. Finally, leveraging diverse forms of content generated by MoM, we develop a three-layer document memory retrieval mechanism, which is grounded in our theoretical proof from the perspective of probabilistic modeling. Extensive experimental results across three distinct domains demonstrate that the MoM framework not only resolves text chunking challenges in existing RAG systems, providing LLMs with semantically complete document memories, but also paves the way for SLMs to achieve human-centric intelligent text processing.