# TRAQ: Trustworthy Retrieval Augmented Question Answering via Conformal Prediction

arXiv: https://arxiv.org/abs/2307.04642 · HTML: https://mlsanity.com/paper/2307.04642/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 20 of its 47 references that are in the map.
- First posted on arXiv 7 Jul 2023.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Uncertainty Sets for Image Classifiers using Conformal Prediction](https://mlsanity.com/paper/2009.14193/) (2020)
- [Distribution-Free, Risk-Controlling Prediction Sets](https://mlsanity.com/paper/2101.02703/) (2021)
- [Conformal Prediction with Large Language Models for Multi-Choice Question Answering](https://mlsanity.com/paper/2305.18404/) (2023)
- [Adversarial Retriever-Ranker for dense text retrieval](https://mlsanity.com/paper/2110.03611/) (2021)
- [PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction](https://mlsanity.com/paper/2001.00106/) (2019)
- [PAC Prediction Sets Under Covariate Shift](https://mlsanity.com/paper/2106.09848/) (2021)
- [A Dense Representation Framework for Lexical and Semantic Matching](https://mlsanity.com/paper/2206.09912/) (2022)

## Built on by (most cited first, 25 of 25)

- [Retrieval-Augmented Generation with Graphs (GraphRAG)](https://mlsanity.com/paper/2501.00309/) (2024)
- [A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions](https://mlsanity.com/paper/2412.05563/) (2024)
- [A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions](https://mlsanity.com/paper/2410.12837/) (2024)
- [Conformal Alignment: Knowing When to Trust Foundation Models with Guarantees](https://mlsanity.com/paper/2405.10301/) (2024)
- [Large Language Models Hallucination: A Comprehensive Survey](https://mlsanity.com/paper/2510.06265/) (2025)
- [Knowledge Boundary of Large Language Models: A Survey](https://mlsanity.com/paper/2412.12472/) (2024)
- [Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.15734/) (2025)
- [Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2502.06872/) (2025)
- [Conformal Prediction: A Data Perspective](https://mlsanity.com/paper/2410.06494/) (2024)
- [Conformal Structured Prediction](https://mlsanity.com/paper/2410.06296/) (2024)
- [Conformal Prediction Sets for Deep Generative Models via Reduction to Conformal Regression](https://mlsanity.com/paper/2503.10512/) (2025)
- [FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees](https://mlsanity.com/paper/2411.02603/) (2024)
- [Principled Context Engineering for RAG: Statistical Guarantees via Conformal Prediction](https://mlsanity.com/paper/2511.17908/) (2025)
- [Quasar: A Programming Language Specialized for LLM Code Actions](https://mlsanity.com/paper/2506.12202/) (2025)
- [Statistical Methods in Generative AI](https://mlsanity.com/paper/2509.07054/) (2025)
- [Federated Language Models Under Bandwidth Budgets: Distillation Rates and Conformal Coverage](https://mlsanity.com/paper/2605.09986/) (2026)
- [Robust Uncertainty Quantification for Self-Evolving Large Language Models via Continual Domain Pretraining](https://mlsanity.com/paper/2510.22931/) (2025)
- [Certified Adaptive Refresh: Anytime-Valid Monitoring for Federated Conformal RAG](https://mlsanity.com/paper/2605.29139/) (2026)
- [Evaluation of Attribution Bias in Generator-Aware Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2410.12380/) (2024)
- [Anchored Confabulation: Partial Evidence Non-Monotonically Amplifies Confident Hallucination in LLMs](https://mlsanity.com/paper/2604.25931/) (2026)
- [Conformal Path Reasoning: Trustworthy Knowledge Graph Question Answering via Path-Level Calibration](https://mlsanity.com/paper/2605.08077/) (2026)
- [Conformalized Large Language Models under Configuration Shift](https://mlsanity.com/paper/2608.01460/) (2026)
- [Distribution-informed Efficient Conformal Prediction for Full Ranking](https://mlsanity.com/paper/2601.23128/) (2026)
- [Cascading Hallucination in Agentic RAG: The CHARM Framework for Detection and Mitigation](https://mlsanity.com/paper/2606.04435/) (2026)
- [Unanimously Wrong: Certified Abstention from How Medical LLM Consensus Forms](https://mlsanity.com/paper/2610.07570/) (2026)

## Abstract (arXiv)

When applied to open-domain question answering, large language models (LLMs) frequently generate incorrect responses based on made-up facts, which are called textithallucinations. Retrieval augmented generation (RAG) is a promising strategy to avoid hallucinations, but it does not provide guarantees on its correctness. To address this challenge, we propose the Trustworthy Retrieval Augmented Question Answering, or textitTRAQ, which provides the first end-to-end statistical correctness guarantee for RAG. TRAQ uses conformal prediction, a statistical technique for constructing prediction sets that are guaranteed to contain the semantically correct response with high probability. Additionally, TRAQ leverages Bayesian optimization to minimize the size of the constructed sets. In an extensive experimental evaluation, we demonstrate that TRAQ provides the desired correctness guarantee while reducing prediction set size by 16.2% on average compared to an ablation. The implementation is available at hrefhttps://github.com/shuoli90/TRAQ.gitTRAQ.