Data to 14 Oct 2026

TRAQ: Trustworthy Retrieval Augmented Question Answering via Conformal Prediction

arXiv 2307.04642 ↗First posted 7 Jul 2023cs.CL cs.AIcited by 25

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

When applied to open-domain question answering, large language models (LLMs) frequently generate incorrect responses based on made-up facts, which are called textithallucinations. Retrieval augmented generation (RAG) is a promising strategy to avoid hallucinations, but it does not provide guarantees on its correctness. To address this challenge, we propose the Trustworthy Retrieval Augmented Question Answering, or textitTRAQ, which provides the first end-to-end statistical correctness guarantee for RAG. TRAQ uses conformal prediction, a statistical technique for constructing prediction sets that are guaranteed to contain the semantically correct response with high probability. Additionally, TRAQ leverages Bayesian optimization to minimize the size of the constructed sets. In an extensive experimental evaluation, we demonstrate that TRAQ provides the desired correctness guarantee while reducing prediction set size by 16.2% on average compared to an ablation. The implementation is available at hrefhttps://github.com/shuoli90/TRAQ.gitTRAQ.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 20 of its 47 references that are in the map. First posted on arXiv 7 Jul 2023.

Builds on

20 of 47 references.

References · 20

2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
2022Training language models to follow instructions with human feedbackcited by 15,971
2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792

8 more

2018A Tutorial on Bayesian Optimizationcited by 744
2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 366
2021Distribution-Free, Risk-Controlling Prediction Setscited by 288
2023Conformal Prediction with Large Language Models for Multi-Choice Question Answeringcited by 139
2021Adversarial Retriever-Ranker for dense text retrievalcited by 83
2019PAC Confidence Sets for Deep Neural Networks via Calibrated Predictioncited by 60
2021PAC Prediction Sets Under Covariate Shiftcited by 45
2022A Dense Representation Framework for Lexical and Semantic Matchingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 8; 2026: 8.

25 papers.

Sort:

By area:

2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 172
2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 121
2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directionscited by 70
2024Conformal Alignment: Knowing When to Trust Foundation Models with Guaranteescited by 54
2025Large Language Models Hallucination: A Comprehensive Surveycited by 49
2024Knowledge Boundary of Large Language Models: A Surveycited by 40
2025Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generationcited by 36
2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 30
2024Conformal Prediction: A Data Perspectivecited by 25
2024Conformal Structured Predictioncited by 10
2025Conformal Prediction Sets for Deep Generative Models via Reduction to Conformal Regressioncited by 10
2024FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guaranteescited by 7

13 more

2025Principled Context Engineering for RAG: Statistical Guarantees via Conformal Predictioncited by 6
2025Quasar: A Programming Language Specialized for LLM Code Actionscited by 4
2025Statistical Methods in Generative AIcited by 4
2026Federated Language Models Under Bandwidth Budgets: Distillation Rates and Conformal Coveragecited by 3
2025Robust Uncertainty Quantification for Self-Evolving Large Language Models via Continual Domain Pretrainingcited by 2
2026Certified Adaptive Refresh: Anytime-Valid Monitoring for Federated Conformal RAGcited by 2
2024Evaluation of Attribution Bias in Generator-Aware Retrieval-Augmented Large Language Modelscited by 1
2026Anchored Confabulation: Partial Evidence Non-Monotonically Amplifies Confident Hallucination in LLMscited by 1
2026Conformal Path Reasoning: Trustworthy Knowledge Graph Question Answering via Path-Level Calibrationcited by 1
2026Conformalized Large Language Models under Configuration Shiftcited by 1
2026Distribution-informed Efficient Conformal Prediction for Full Rankingcited by 0
2026Cascading Hallucination in Agentic RAG: The CHARM Framework for Detection and Mitigationcited by 0
2026Unanimously Wrong: Certified Abstention from How Medical LLM Consensus Formscited by 0

Similar, not cited

Close in wording, no citation either way