# PRCA: Fitting Black-Box Large Language Models for Retrieval Question Answering via Pluggable Reward-Driven Contextual Adapter

arXiv: https://arxiv.org/abs/2310.18347 · HTML: https://mlsanity.com/paper/2310.18347/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 30 of its 36 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 30 of 30)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales](https://mlsanity.com/paper/2308.01320/) (2023)
- [TopiOCQA: Open-domain Conversational Question Answering with Topic Switching](https://mlsanity.com/paper/2110.00768/) (2021)
- [RetrievalSum: A Retrieval Enhanced Framework for Abstractive Summarization](https://mlsanity.com/paper/2109.07943/) (2021)
- [Improving Question Answering with External Knowledge](https://mlsanity.com/paper/1902.00993/) (2019)

## Built on by (most cited first, 30 of 38)

- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey](https://mlsanity.com/paper/2311.07914/) (2023)
- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [Trustworthiness in Retrieval-Augmented Generation Systems: A Survey](https://mlsanity.com/paper/2409.10102/) (2024)
- [A Survey on Retrieval-Augmented Text Generation for Large Language Models](https://mlsanity.com/paper/2404.10981/) (2024)
- [Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely](https://mlsanity.com/paper/2409.14924/) (2024)
- [CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models](https://mlsanity.com/paper/2401.17043/) (2024)
- [Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks](https://mlsanity.com/paper/2407.21059/) (2024)
- [Knowledge Boundary of Large Language Models: A Survey](https://mlsanity.com/paper/2412.12472/) (2024)
- [Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2502.06872/) (2025)
- [BIDER: Bridging Knowledge Inconsistency for Efficient Retrieval-Augmented LLMs via Key Supporting Evidence](https://mlsanity.com/paper/2402.12174/) (2024)
- [RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing](https://mlsanity.com/paper/2404.19543/) (2024)
- [A Survey of Conversational Search](https://mlsanity.com/paper/2410.15576/) (2024)
- [Characterizing Prompt Compression Methods for Long Context Inference](https://mlsanity.com/paper/2407.08892/) (2024)
- [An Information Bottleneck Perspective for Effective Noise Filtering on Retrieval-Augmented Generation](https://mlsanity.com/paper/2406.01549/) (2024)
- [Refiner: Restructure Retrieval Content Efficiently to Advance Question-Answering Capabilities](https://mlsanity.com/paper/2406.11357/) (2024)
- [GINGER: Grounded Information Nugget-Based Generation of Responses](https://mlsanity.com/paper/2503.18174/) (2025)
- [Hierarchical Document Refinement for Long-context Retrieval-augmented Generation](https://mlsanity.com/paper/2505.10413/) (2025)
- [Long Context In-Context Compression by Getting to the Gist of Gisting](https://mlsanity.com/paper/2504.08934/) (2025)
- [RE-RAG: Improving Open-Domain QA Performance and Interpretability with Relevance Estimator in Retrieval-Augmented Generation](https://mlsanity.com/paper/2406.05794/) (2024)
- [Preference-Guided Refactored Tuning for Retrieval Augmented Code Generation](https://mlsanity.com/paper/2409.15895/) (2024)
- [CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation Generation](https://mlsanity.com/paper/2410.23090/) (2024)
- [SCOPE: A Generative Approach for LLM Prompt Compression](https://mlsanity.com/paper/2508.15813/) (2025)
- [UniOQA: A Unified Framework for Knowledge Graph Question Answering with Large Language Models](https://mlsanity.com/paper/2406.02110/) (2024)
- [CompLLM: Compression for Long Context Q&A](https://mlsanity.com/paper/2509.19228/) (2025)
- [Leveraging LLM-Assisted Query Understanding for Live Retrieval-Augmented Generation](https://mlsanity.com/paper/2506.21384/) (2025)
- [Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2411.07820/) (2024)
- [LLM-Ref: Enhancing Reference Handling in Technical Writing with Large Language Models](https://mlsanity.com/paper/2411.00294/) (2024)
- [K-COMP: Retrieval-Augmented Medical Domain Question Answering With Knowledge-Injected Compressor](https://mlsanity.com/paper/2501.13567/) (2025)
- [Addressing the sustainable AI trilemma: a case study on LLM agents and RAG](https://mlsanity.com/paper/2501.08262/) (2025)

## Abstract (arXiv)

The Retrieval Question Answering (ReQA) task employs the retrieval-augmented framework, composed of a retriever and generator. The generator formulates the answer based on the documents retrieved by the retriever. Incorporating Large Language Models (LLMs) as generators is beneficial due to their advanced QA capabilities, but they are typically too large to be fine-tuned with budget constraints while some of them are only accessible via APIs. To tackle this issue and further improve ReQA performance, we propose a trainable Pluggable Reward-Driven Contextual Adapter (PRCA), keeping the generator as a black box. Positioned between the retriever and generator in a Pluggable manner, PRCA refines the retrieved information by operating in a token-autoregressive strategy via maximizing rewards of the reinforcement learning phase. Our experiments validate PRCA's effectiveness in enhancing ReQA performance on three datasets by up to 20% improvement to fit black-box LLMs into existing frameworks, demonstrating its considerable potential in the LLMs era.