# Embedding-Informed Adaptive Retrieval-Augmented Generation of Large Language Models

arXiv: https://arxiv.org/abs/2404.03514 · HTML: https://mlsanity.com/paper/2404.03514/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation.
- Builds on 14 of its 20 references that are in the map.
- First posted on arXiv 4 Apr 2024.

## Placed in

- Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/)

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Representation Degeneration Problem in Training Natural Language Generation Models](https://mlsanity.com/paper/1907.12009/) (2019)
- [Do Large Language Models Know What They Don't Know?](https://mlsanity.com/paper/2305.18153/) (2023)
- [A Survey on Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2202.01110/) (2022)
- [Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation](https://mlsanity.com/paper/2307.11019/) (2023)
- [Unsupervised Cross-Task Generalization via Retrieval Augmentation](https://mlsanity.com/paper/2204.07937/) (2022)
- [Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications](https://mlsanity.com/paper/2311.05876/) (2023)

## Built on by (most cited first, 9 of 9)

- [BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models](https://mlsanity.com/paper/2406.11675/) (2024)
- [Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning](https://mlsanity.com/paper/2410.14211/) (2024)
- [PruneRAG: Confidence-Guided Query Decomposition Trees for Efficient Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.11024/) (2026)
- [RAG-VisualRec: An Open Resource for Vision- and Text-Enhanced Retrieval-Augmented Generation in Recommendation](https://mlsanity.com/paper/2506.20817/) (2025)
- [PRoH: Dynamic Planning and Reasoning over Knowledge Hypergraphs for Retrieval-Augmented Generation](https://mlsanity.com/paper/2510.12434/) (2025)
- [MEG-RAG: Quantifying Multi-modal Evidence Grounding for Evidence Selection in RAG](https://mlsanity.com/paper/2604.24564/) (2026)
- [Purifying Multimodal Retrieval: Fragment-Level Evidence Selection for RAG](https://mlsanity.com/paper/2604.27600/) (2026)
- [Learning to Retrieve Navigable Candidates for Efficient Vision-and-Language Navigation](https://mlsanity.com/paper/2602.15724/) (2026)
- [AUSO: Action-Level Unified Skill Optimization from Internalization to Utilization](https://mlsanity.com/paper/2608.21292/) (2026)

## Abstract (arXiv)

Retrieval-augmented large language models (LLMs) have been remarkably competent in various NLP tasks. However, it was observed by previous works that retrieval is not always helpful, especially when the LLM is already knowledgeable on the query to answer. Motivated by this, Adaptive Retrieval-Augmented Generation (ARAG) studies retrieving only when the knowledge asked by the query is absent in the LLM. Previous works of ARAG either require accessing the pre-training corpus or prompting with additional model inferences. Aiming to avoid such drawbacks, we propose to determine whether the model is knowledgeable on a query via inspecting the (contextualized) pre-trained token embeddings of LLMs. We hypothesize that such embeddings capture rich information on the model's intrinsic knowledge base, which enables an efficient way of judging the necessity to retrieve from an external corpus. Extensive experiments demonstrate our ARAG approach's superior performance across various benchmarks.