# MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA

arXiv: https://arxiv.org/abs/2606.05749 · HTML: https://mlsanity.com/paper/2606.05749/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 4 Jun 2026.

## Builds on (most cited first, 24 of 24)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [Memory in the Age of AI Agents](https://mlsanity.com/paper/2512.13564/) (2025)
- [InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition](https://mlsanity.com/paper/2309.15112/) (2023)
- [Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG](https://mlsanity.com/paper/2501.09136/) (2025)
- [VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents](https://mlsanity.com/paper/2410.10594/) (2024)
- [Chain of Agents: Large Language Models Collaborating on Long-Context Tasks](https://mlsanity.com/paper/2406.02818/) (2024)
- [MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizations](https://mlsanity.com/paper/2407.01523/) (2024)
- [M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding](https://mlsanity.com/paper/2411.04952/) (2024)
- [MDocAgent: A Multi-Modal Multi-Agent Framework for Document Understanding](https://mlsanity.com/paper/2503.13964/) (2025)
- [DOCBENCH: A Benchmark for Evaluating LLM-based Document Reading Systems](https://mlsanity.com/paper/2407.10701/) (2024)
- [SV-RAG: LoRA-Contextualizing Adaptation of MLLMs for Long Document Understanding](https://mlsanity.com/paper/2411.01106/) (2024)
- [Docopilot: Improving Multimodal Models for Document-Level Understanding](https://mlsanity.com/paper/2507.14675/) (2025)
- [Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understanding](https://mlsanity.com/paper/2511.22850/) (2025)
- [MoLoRAG: Bootstrapping Document Understanding via Multi-modal Logic-aware Retrieval](https://mlsanity.com/paper/2509.07666/) (2025)
- [BookRAG: A Hierarchical Structure-aware Index-based Approach for Retrieval-Augmented Generation on Complex Documents](https://mlsanity.com/paper/2512.03413/) (2025)
- [DocLens : A Tool-Augmented Multi-Agent Framework for Long Visual Document Understanding](https://mlsanity.com/paper/2511.11552/) (2025)
- [DocDancer: Towards Agentic Document-Grounded Information Seeking](https://mlsanity.com/paper/2601.05163/) (2026)
- [DocXChain: A Powerful Open-Source Toolchain for Document Parsing and Beyond](https://mlsanity.com/paper/2310.12430/) (2023)
- [DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding](https://mlsanity.com/paper/2604.12812/) (2026)

## Built on by (most cited first, 2 of 2)

- [Collaborative Memory for Multi-Agent VLM Systems](https://mlsanity.com/paper/2609.17921/) (2026)
- [Locate, Judge, Read: Role-Specialized Tool Agents over a Knowledge-Point Index for Open-Domain Multimodal Document QA](https://mlsanity.com/paper/2610.13128/) (2026)

## Abstract (arXiv)

Iterative retrieval-reasoning agents have recently shown promise for multimodal long-document question answering. However, most existing systems maintain a single growing context that mixes retrieval traces, observations, and intermediate reasoning. As interactions accumulate, key evidence becomes scattered and diluted, making multi-hop reasoning noisy. We propose MARDoc, a Memory-Aware Refinement Agent framework that decouples long-document QA into three specialized agents: an Explorer for multi-granularity multimodal retrieval, a Refiner for distilling interaction traces into structured evidence and reasoning memories, and a Reflector for checking evidence sufficiency and providing targeted feedback. Across iterations, the agents rely on a dynamically updated structured memory rather than a full accumulated interaction history. This design reduces context noise while preserving answer-critical facts and their logical dependencies. Experiments on MMLongBench-Doc and DocBench show that MARDoc achieves strong results, outperforming same-backbone baselines and demonstrating the effectiveness of structured memory for agentic document QA.