mlsanityData to 14 Oct 2026

Assessing RAG and HyDE on 1B vs. 4B-Parameter Gemma LLMs for Personal Assistants Integretion

arXiv 2506.21568 ↗First posted 12 Jun 2025cs.CLcited by 2

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

Resource efficiency is a critical barrier to deploying large language models (LLMs) in edge and privacy-sensitive applications. This study evaluates the efficacy of two augmentation strategies--Retrieval-Augmented Generation (RAG) and Hypothetical Document Embeddings (HyDE)--on compact Gemma LLMs of 1 billion and 4 billion parameters, within the context of a privacy-first personal assistant. We implement short-term memory via MongoDB and long-term semantic storage via Qdrant, orchestrated through FastAPI and LangChain, and expose the system through a React.js frontend. Across both model scales, RAG consistently reduces latency by up to 17% and eliminates factual hallucinations when responding to user-specific and domain-specific queries. HyDE, by contrast, enhances semantic relevance--particularly for complex physics prompts--but incurs a 25--40% increase in response time and a non-negligible hallucination rate in personal-data retrieval. Comparing 1 B to 4 B models, we observe that scaling yields marginal throughput gains for baseline and RAG pipelines, but magnifies HyDE's computational overhead and variability. Our findings position RAG as the pragmatic choice for on-device personal assistants powered by small-scale LLMs.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 5 of its 7 references that are in the map. First posted on arXiv 12 Jun 2025.

Builds on

5 of 7 references.

References · 5

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Modelscited by 77

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Models for minimalist RAG: B1ade 335M Embedding and 1B Parameter Small Language Modelscited by 1
  2. 2026Multimodal Hybrid Retrieval-Augmented Generation for Scientific Document Understanding using Open-Source SLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systems
  2. 2025Private-RAG: Answering Multiple Queries with LLMs while Keeping Your Data Private
  3. 2024Comparing Retrieval-Augmentation and Parameter-Efficient Fine-Tuning for Privacy-Preserving Personalization of Large Language Models
  4. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Survey
  5. 2026Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systems
  6. 2024RAG with Differential Privacy
  7. 2024No Free Lunch: Retrieval-Augmented Generation Undermines Fairness in LLMs, Even for Vigilant Users
  8. 2024RemoteRAG: A Privacy-Preserving LLM Cloud RAG Service