# MeMemo: On-device Retrieval Augmentation for Private and Personalized Text Generation

arXiv: https://arxiv.org/abs/2407.01972 · HTML: https://mlsanity.com/paper/2407.01972/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 33 of its 72 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 30 of 33)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [Text and Code Embeddings by Contrastive Pre-Training](https://mlsanity.com/paper/2201.10005/) (2022)
- [The Power of Noise: Redefining Retrieval for RAG Systems](https://mlsanity.com/paper/2401.14887/) (2024)
- [Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs](https://mlsanity.com/paper/2312.05934/) (2023)
- [DocPrompting: Generating Code by Retrieving the Docs](https://mlsanity.com/paper/2207.05987/) (2022)
- [RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture](https://mlsanity.com/paper/2401.08406/) (2024)
- [A Retrieve-and-Edit Framework for Predicting Structured Outputs](https://mlsanity.com/paper/1812.01194/) (2018)
- [TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usage](https://mlsanity.com/paper/2308.03427/) (2023)
- [EdgeRec: Recommender System on Edge in Mobile Taobao](https://mlsanity.com/paper/2005.08416/) (2020)
- [WizMap: Scalable Interactive Visualization for Exploring Large Machine Learning Embeddings](https://mlsanity.com/paper/2306.09328/) (2023)
- [Opportunities for Retrieval and Tool Augmented Large Language Models in Scientific Facilities](https://mlsanity.com/paper/2312.01291/) (2023)
- [Passage Retrieval for Outside-Knowledge Visual Question Answering](https://mlsanity.com/paper/2105.03938/) (2021)
- [Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values](https://mlsanity.com/paper/2206.15465/) (2022)
- [CoPrompt: Supporting Prompt Sharing and Referring in Collaborative Natural Language Programming](https://mlsanity.com/paper/2310.09235/) (2023)
- [Rethinking Privacy in Machine Learning Pipelines from an Information Flow Control Perspective](https://mlsanity.com/paper/2311.15792/) (2023)
- [Gender, Age, and Technology Education Influence the Adoption and Appropriation of LLMs](https://mlsanity.com/paper/2310.06556/) (2023)
- [Amortized Neural Networks for Low-Latency Speech Recognition](https://mlsanity.com/paper/2108.01553/) (2021)
- [GAM Coach: Towards Interactive and User-centered Algorithmic Recourse](https://mlsanity.com/paper/2302.14165/) (2023)
- [Wordflow: Social Prompt Engineering for Large Language Models](https://mlsanity.com/paper/2401.14447/) (2024)
- [Efficient On-Device Session-Based Recommendation](https://mlsanity.com/paper/2209.13422/) (2022)
- [Adapting LLMs for Efficient, Personalized Information Retrieval: Methods and Implications](https://mlsanity.com/paper/2311.12287/) (2023)
- [GPU-based Private Information Retrieval for On-Device Machine Learning Inference](https://mlsanity.com/paper/2301.10904/) (2023)

## Built on by (most cited first, 13 of 13)

- [A Survey of Personalization: From RAG to Agent](https://mlsanity.com/paper/2504.10147/) (2025)
- [MobileRAG: A Fast, Memory-Efficient, and Energy-Efficient Method for On-Device RAG](https://mlsanity.com/paper/2507.01079/) (2025)
- [TeleRAG: Efficient Retrieval-Augmented Generation Inference with Lookahead Retrieval](https://mlsanity.com/paper/2502.20969/) (2025)
- [EnronQA: Towards Personalized RAG over Private Documents](https://mlsanity.com/paper/2505.00263/) (2025)
- [SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2601.03979/) (2026)
- [Accelerating Mobile Language Model via Speculative Decoding and NPU-Coordinated Execution](https://mlsanity.com/paper/2510.15312/) (2025)
- [LEANN: A Low-Storage Vector Index](https://mlsanity.com/paper/2506.08276/) (2025)
- [WebANNS: Fast and Efficient Approximate Nearest Neighbor Search in Web Browsers](https://mlsanity.com/paper/2507.00521/) (2025)
- [On-device Semantic Selection Made Low Latency and Memory Efficient with Monolithic Forwarding](https://mlsanity.com/paper/2510.15620/) (2025)
- [TileFuse: A Fused Mixed-Precision Kernel Library for Efficient Quantized LLM Inference on AMD NPUs](https://mlsanity.com/paper/2606.11357/) (2026)
- [Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systems](https://mlsanity.com/paper/2606.25533/) (2026)
- [As We May Search](https://mlsanity.com/paper/2606.29652/) (2026)
- [PixARG: Pixel-Asynchronous Retrieval-Augmented Diffusion for Fast and Realistic Image Generation](https://mlsanity.com/paper/2610.14713/) (2026)

## Abstract (arXiv)

Retrieval-augmented text generation (RAG) addresses the common limitations of large language models (LLMs), such as hallucination, by retrieving information from an updatable external knowledge base. However, existing approaches often require dedicated backend servers for data storage and retrieval, thereby limiting their applicability in use cases that require strict data privacy, such as personal finance, education, and medicine. To address the pressing need for client-side dense retrieval, we introduce MeMemo, the first open-source JavaScript toolkit that adapts the state-of-the-art approximate nearest neighbor search technique HNSW to browser environments. Developed with modern and native Web technologies, such as IndexedDB and Web Workers, our toolkit leverages client-side hardware capabilities to enable researchers and developers to efficiently search through millions of high-dimensional vectors in the browser. MeMemo enables exciting new design and research opportunities, such as private and personalized content creation and interactive prototyping, as demonstrated in our example application RAG Playground. Reflecting on our work, we discuss the opportunities and challenges for on-device dense retrieval. MeMemo is available at https://github.com/poloclub/mememo.