mlsanityData to 14 Oct 2026

MobileRAG: A Fast, Memory-Efficient, and Energy-Efficient Method for On-Device RAG

arXiv 2507.01079 ↗First posted 1 Jul 2025cs.DBcited by 11

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) has proven effective on server infrastructures, but its application on mobile devices is still underexplored due to limited memory and power resources. Existing vector search and RAG solutions largely assume abundant computation resources, making them impractical for on-device scenarios. In this paper, we propose MobileRAG, a fully on-device pipeline that overcomes these limitations by combining a mobile-friendly vector search algorithm, textitEcoVector, with a lightweight textitSelective Content Reduction (SCR) method. By partitioning and partially loading index data, EcoVector drastically reduces both memory footprint and CPU usage, while the SCR method filters out irrelevant text to diminish Language Model (LM) input size without degrading accuracy. Extensive experiments demonstrated that MobileRAG significantly outperforms conventional vector search and RAG methods in terms of latency, memory usage, and power consumption, while maintaining accuracy and enabling offline operation to safeguard privacy in resource-constrained environments.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 27 of its 57 references that are in the map. First posted on arXiv 1 Jul 2025.

Builds on

27 of 57 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024EdgeRAG: Online-Indexed RAG for Edge Devices30 Dec
  2. 2024Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks26 Jul
  3. 2024MeMemo: On-device Retrieval Augmentation for Private and Personalized Text Generation2 Jul
  4. 2023Retrieval-Augmented Generation for Large Language Models: A Survey18 Dec

Other references · 23

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  8. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  9. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  10. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  11. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  12. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
11 more
  1. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  2. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 482
  3. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 332
  4. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 223
  5. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 202
  6. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 172
  7. 2018ANN-Benchmarks: A Benchmarking Tool for Approximate Nearest Neighbor Algorithmscited by 130
  8. 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 128
  9. 2021A Comprehensive Survey and Experimental Comparison of Graph-Based Approximate Nearest Neighbor Searchcited by 82
  10. 2021Hindsight: Posterior-guided training of retrievers for improved open-ended generationcited by 38
  11. 2016EFANNA : An Extremely Fast Approximate Nearest Neighbor Search Algorithm Based on kNN Graphcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 10.

11 papers.

  1. 2026Embodied Foundation Models at the Edge: A Survey of Deployment Constraints and Mitigation Strategiescited by 9
  2. 2025Accelerating Mobile Language Model via Speculative Decoding and NPU-Coordinated Executioncited by 5
  3. 2026From Volume to Value: Preference-Aligned Memory Construction for On-Device RAGcited by 1
  4. 2026Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Mergingcited by 1
  5. 2026A Unified Model and Document Representation for On-Device Retrieval-Augmented Generationcited by 0
  6. 2026Accelerating AI-Powered Research: The PuppyChatter Framework for Usable and Flexible Toolingcited by 0
  7. 2026One Token per Multimodal Evidence: Latent Memory for Resource-Constrained QAcited by 0
  8. 2026From Retrieved Context to Runtime Control: Adaptive Compression for Edge-based RAGcited by 0
  9. 2026TierKV: Long-Context On-Device LLMs via Predictive Multi-Tier KV Cachingcited by 0
  10. 2026Dynamic Flow, Static Graph: KV Cache Reuse for Efficient LLM Serving on Mobile NPUscited by 0
  11. 2026PhaseGate: Phase-Aware CPU Retrieval Scheduling for On-Device LLMs on Unified Memorycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025PerCache: Predictive Hierarchical Cache for RAG Applications on Mobile Devices
  2. 2026Energy-Efficient On-Device RAG on a Mobile NPU: System Design and Benchmark on Snapdragon X Elite
  3. 2025Engineering the RAG Stack: A Comprehensive Review of the Architecture and Trust Frameworks for Retrieval-Augmented Generation Systems
  4. 2025MobileRAG: Enhancing Mobile Agent with Retrieval-Augmented Generation
  5. 2025RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving
  6. 2025MicroNN: An On-device Disk-resident Updatable Vector Database
  7. 2025FlexRAG: A Flexible and Comprehensive Framework for Retrieval-Augmented Generation
  8. 20255G Network Automation Using Local Large Language Models and Retrieval-Augmented Generation