mlsanityData to 14 Oct 2026

ChatQA: Surpassing GPT-4 on Conversational QA and RAG

arXiv 2401.10225 ↗First posted 18 Jan 2024cs.CL cs.AI cs.IRcited by 90

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

In this work, we introduce ChatQA, a suite of models that outperform GPT-4 on retrieval-augmented generation (RAG) and conversational question answering (QA). To enhance generation, we propose a two-stage instruction tuning method that significantly boosts the performance of RAG. For effective retrieval, we introduce a dense retriever optimized for conversational QA, which yields results comparable to the alternative state-of-the-art query rewriting models, while substantially reducing deployment costs. We also present the ChatRAG Bench, which encompasses ten datasets covering comprehensive evaluations on RAG, table-related QA, arithmetic calculations, and scenarios involving unanswerable questions. Our ChatQA-1.0-70B (score: 54.14), built on Llama2, a weaker foundation model than GPT-4, can slightly outperform GPT-4-0613 (score: 53.90) and GPT-4-Turbo-2024-04-09 (score: 54.03) on the ChatRAG Bench, without relying on any synthetic data from OpenAI GPT models. Notably, the Llama3-ChatQA-1.5-70B model surpasses the accuracy of GPT-4-Turbo-2024-04-09, achieving a 4.4% improvement. To advance research in this field, we open-sourced the model weights, instruction tuning data, ChatRAG Bench, and retriever for the community: https://chatqa-project.github.io/.

Key facts. Cited by 90 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 68 of its 92 references that are in the map. First posted on arXiv 18 Jan 2024.

Builds on

68 of 92 references.

References · 68

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  10. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  11. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  12. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
38 more
  1. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  2. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  3. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  4. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  5. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  6. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  7. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  8. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  9. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
  10. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  11. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  12. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  13. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  14. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  15. 2018CoQA: A Conversational Question Answering Challengecited by 683
  16. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  17. 2017The NarrativeQA Reading Comprehension Challengecited by 597
  18. 2019ELI5: Long Form Question Answeringcited by 575
  19. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
  20. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 515
  21. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 408
  22. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 408
  23. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 397
  24. 2018QuAC : Question Answering in Contextcited by 381
  25. 2016NewsQA: A Machine Comprehension Datasetcited by 331
  26. 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 236
  27. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 195
  28. 2022SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualizationcited by 168
  29. 2020Open-Domain Question Answering Goes Conversational via Question Rewritingcited by 148
  30. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 118
  31. 2023Retrieval meets Long Context Large Language Modelscited by 115
  32. 2019Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoningcited by 100
  33. 2020Open-Retrieval Conversational Question Answeringcited by 99
  34. 2021TopiOCQA: Open-domain Conversational Question Answering with Topic Switchingcited by 96
  35. 2020Question Rewriting for Conversational Question Answeringcited by 93
  36. 2020doc2dial: A Goal-Oriented Document-Grounded Dialogue Datasetcited by 88
  37. 2020Few-Shot Generative Conversational Query Rewritingcited by 85
  38. 2019Reasoning Over Paragraph Effects in Situationscited by 79

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 25; 2025: 41; 2026: 24.

90 papers.

Sort:

By area:

  1. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
  2. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 202
  3. 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 197
  4. 2024Compact Language Models via Pruning and Knowledge Distillationcited by 148
  5. 2024Nemotron-4 340B Technical Reportcited by 144
  6. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 116
  7. 2024Searching for Best Practices in Retrieval-Augmented Generationcited by 83
  8. 2024LLM Pruning and Distillation in Practice: The Minitron Approachcited by 82
  9. 2025Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learningcited by 72
  10. 2024Inference Scaling for Long-Context Retrieval Augmented Generationcited by 65
  11. 2024ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilitiescited by 53
  12. 2025RocketEval: Efficient Automated LLM Evaluation via Grading Checklistcited by 39
38 more
  1. 2024PEDANTS: Cheap but Effective and Interpretable Answer Equivalencecited by 31
  2. 2024SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domainscited by 28
  3. 2025FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generationcited by 28
  4. 2025MA-RAG: Multi-Agent Retrieval-Augmented Generation via Collaborative Chain-of-Thought Reasoningcited by 26
  5. 2024RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructionscited by 25
  6. 2024Efficient Knowledge Injection in LLMs via Self-Distillationcited by 24
  7. 2024RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignmentcited by 23
  8. 2025Kanana: Compute-efficient Bilingual Language Modelscited by 20
  9. 2025Collab-RAG: Boosting Retrieval-Augmented Generation for Complex Question Answering via White-Box and Black-Box LLM Collaborationcited by 20
  10. 2024Adaptive Query Rewriting: Aligning Rewriters through Marginal Probability of Conversational Answerscited by 15
  11. 2025DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generationcited by 15
  12. 2024DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answeringcited by 14
  13. 2024OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMscited by 14
  14. 2024MIRAGE-Bench: Automatic Multilingual Benchmark Arena for Retrieval-Augmented Generation Systemscited by 14
  15. 2025Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAGcited by 14
  16. 2026Coding Agents are Effective Long-Context Processorscited by 12
  17. 2026GraphPlanner: Graph Memory-Augmented Agentic Routing for Multi-Agent LLMscited by 12
  18. 2025LevelRAG: Enhancing Retrieval-Augmented Generation with Multi-hop Logic Planning over Rewriting Augmented Searcherscited by 11
  19. 2025Iterative Self-Incentivization Empowers Large Language Models as Agentic Searcherscited by 11
  20. 2025CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoningcited by 11
  21. 2025FusionFactory: Fusing LLM Capabilities with Multi-LLM Log Datacited by 10
  22. 2026SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decodingcited by 10
  23. 2025SearchRAG: Can Search Engines Be Helpful for LLM-based Medical Question Answering?cited by 9
  24. 2025Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Modelscited by 9
  25. 2025SelfAug: Mitigating Catastrophic Forgetting in Retrieval-Augmented Generation via Distribution Self-Alignmentcited by 9
  26. 2024Phase Transitions in the Output Distribution of Large Language Modelscited by 7
  27. 2024Reward-RAG: Enhancing RAG with Reward Driven Supervisioncited by 7
  28. 2025ASTRAL: Automated Safety Testing of Large Language Modelscited by 7
  29. 2025Single-Pass Document Scanning for Question Answeringcited by 6
  30. 2025HIRAG: Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generationcited by 6
  31. 2024RAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented Generationcited by 5
  32. 2025IPR: Intelligent Prompt Routing with User-Controlled Quality-Cost Trade-offscited by 5
  33. 2024Retrieval Augmented Instruction Tuning for Open NER with Large Language Modelscited by 4
  34. 2024Combining LLMs and Knowledge Graphs to Reduce Hallucinations in Question Answeringcited by 4
  35. 2025OpenRAG: Optimizing RAG End-to-End via In-Context Retrieval Learningcited by 4
  36. 2025BioMol-MQA: A Multi-Modal Question Answering Dataset For LLM Reasoning Over Bio-Molecular Interactionscited by 4
  37. 2025AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Modelcited by 4
  38. 2025Retrieval Augmented Generation Evaluation for Health Documentscited by 3

50 most cited of 90 All 90 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Boosting Conversational Question Answering with Fine-Grained Retrieval-Augmentation and Self-Check
  2. 2026EverydayGPT: Confidence-Gated Routing for Efficient and Safe Hybrid GPT-RAG Conversational QA
  3. 2025T2-RAGBench: Text-and-Table Benchmark for Evaluating Retrieval-Augmented Generation
  4. 2024RAGONITE: Iterative Retrieval on Induced Databases and Verbalized RDF for Conversational QA over KGs with RAG
  5. 2024W-RAG: Weakly Supervised Dense Retrieval in RAG for Open-domain Question Answering
  6. 2024Learning When to Retrieve, What to Rewrite, and How to Respond in Conversational QA
  7. 2025Evaluation of retrieval-based QA on QUEST-LOFT
  8. 2024RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering