mlsanityData to 14 Oct 2026

As We May Search

arXiv 2606.29652 ↗First posted 28 Jun 2026cs.IRcited by 0

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

The sensitive information in personal documents, legal files, and medical records is among the most valuable things to search, yet current retrieval-augmented generation systems still require sending content to remote servers. We propose local-first IR, a design philosophy where indexes, models, and inference reside on user devices, treating remote services as optional. This paper makes four contributions: (1) a framework organizing retrieval architectures along three dimensions: privacy and control, capability, and accessibility, (2) experiments on consumer hardware across five benchmarks, scaling from 1K to 1M documents with dense retrieval, BM25, and hybrid fusion. Dense retrieval keeps over 91% nDCG@10 up to 100K documents, with approximate HNSW indexes extending this to 1M with only 2% quality loss; a 7B local language model reaches within 4 points of a cloud baseline on answer quality, (3) competing perspectives for and against local-first IR, informed by experimental evidence, and (4) a research agenda identifying open problems. The real tradeoff is scope rather than quality: what matters is what you can search, not how well you can search it.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 22 of its 62 references that are in the map. First posted on arXiv 28 Jun 2026.

Builds on

22 of 62 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Efficient Privacy-Preserving Retrieval Augmented Generation with Distance-Preserving Encryption18 Jan
  2. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systems7 Jan
  3. 2025Differentially Private Synthetic Text Generation for Retrieval-Augmented Generation (RAG)8 Oct
  4. 2025HyFedRAG: A Federated Retrieval-Augmented Generation Framework for Heterogeneous and Privacy-Sensitive Data8 Sep
  5. 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)23 Feb

Other references · 17

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2017Billion-scale similarity search with GPUscited by 2,497
  4. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  5. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  6. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  7. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 208
  8. 2023Text Embeddings Reveal (Almost) As Much As Textcited by 192
  9. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 191
  10. 2022The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymizationcited by 95
  11. 2025How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inferencecited by 76
  12. 2024PersonaRAG: Enhancing Retrieval-Augmented Generation Systems with User-Centric Agentscited by 30
5 more
  1. 2024MeMemo: On-device Retrieval Augmentation for Private and Personalized Text Generationcited by 13
  2. 2024Anatomizing Deep Learning Inference in Web Browserscited by 6
  3. 2025Variations in Relevance Judgments and the Shelf Life of Test Collectionscited by 5
  4. 2025FedRAG: A Framework for Fine-Tuning Retrieval-Augmented Generation Systemscited by 4
  5. 2016Ten Blue Links on Marscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026PILLAR: Private Inverted-Index Lexical Lookup for Augmented Retrieval
  2. 2026A Unified Model and Document Representation for On-Device Retrieval-Augmented Generation
  3. 2025Efficient Distributed Retrieval-Augmented Generation for Enhancing Language Model Performance
  4. 2026Pointing the Way, Hiding the Destination: Practical Private Dense Retrieval at Scale
  5. 2024RemoteRAG: A Privacy-Preserving LLM Cloud RAG Service
  6. 2026RAGAL: A Frugal, Fully Local Retrieval-Augmented Assistant for Technical Support at a Government Agency
  7. 2026Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systems
  8. 2023Don't forget private retrieval: distributed private similarity search for large language models