mlsanityData to 7 Oct 2026

A Survey of Large Language Model Agents for Question Answering

arXiv 2503.19213 ↗First posted 24 Mar 2025cs.CL cs.AI cs.HCcited by 46

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

This paper surveys the development of large language model (LLM)-based agents for question answering (QA). Traditional agents face significant limitations, including substantial data requirements and difficulty in generalizing to new environments. LLM-based agents address these challenges by leveraging LLMs as their core reasoning engine. These agents achieve superior QA results compared to traditional QA pipelines and naive LLM QA systems by enabling interaction with external environments. We systematically review the design of LLM agents in the context of QA tasks, organizing our discussion across key stages: planning, question understanding, information retrieval, and answer generation. Additionally, this paper identifies ongoing challenges and explores future research directions to enhance the performance of LLM agent QA systems.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 74 of its 88 references that are in the map. First posted on arXiv 24 Mar 2025.

Builds on

74 of 88 references.

References · 74

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  8. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  9. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  10. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  11. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  12. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
38 more
  1. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  6. 2015Continuous control with deep reinforcement learningcited by 3,279
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  8. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  9. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  10. 2019Language Models as Knowledge Bases?cited by 1,922
  11. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  12. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  13. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  14. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  15. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  16. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  17. 2023Teaching Large Language Models to Self-Debugcited by 963
  18. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  19. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  20. 2022PAL: Program-aided Language Modelscited by 635
  21. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  22. 2019ELI5: Long Form Question Answeringcited by 567
  23. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  24. 2023Active Retrieval Augmented Generationcited by 506
  25. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
  26. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 397
  27. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  28. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
  29. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  30. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
  31. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
  32. 2023FireAct: Toward Language Agent Fine-tuningcited by 247
  33. 2022FOLIO: Natural Language Reasoning with First-Order Logiccited by 233
  34. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  35. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  36. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
  37. 2022Binding Language Models in Symbolic Languagescited by 224
  38. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 211

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 24; 2026: 22.

46 papers.

Sort:

By area:

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  2. 2025LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarioscited by 25
  3. 2025Coarse-to-Fine Grounded Memory for LLM Agent Planningcited by 16
  4. 2025BookRAG: A Hierarchical Structure-aware Index-based Approach for Retrieval-Augmented Generation on Complex Documentscited by 16
  5. 2025PaperArena: An Evaluation Benchmark for Tool-Augmented Agentic Reasoning on Scientific Literaturecited by 15
  6. 2025Humanizing Machines: Rethinking LLM Anthropomorphism Through a Multi-Level Framework of Designcited by 11
  7. 2025LeTS: Learning to Think-and-Search via Process-and-Outcome Reward Hybridizationcited by 10
  8. 2026Exploring Knowledge Purification in Multi-Teacher Knowledge Distillation for LLMscited by 9
  9. 2025S-DAG: A Subject-Based Directed Acyclic Graph for Multi-Agent Heterogeneous Reasoningcited by 8
  10. 2025Enhancing Large Language Model Reasoning with Reward Models: An Analytical Surveycited by 7
  11. 2025Beyond Chunks and Graphs: Retrieval-Augmented Generation through Triplet-Driven Thinkingcited by 6
  12. 2025Injecting Knowledge Graphs into Large Language Modelscited by 5
34 more
  1. 2025BioMedSearch: A Multi-Source Biomedical Retrieval Framework Based on LLMscited by 4
  2. 2025Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysiscited by 3
  3. 2025Select to Know: An Internal-External Knowledge Self-Selection Framework for Domain-Specific Question Answeringcited by 3
  4. 2025"To Survive, I Must Defect": Jailbreaking LLMs via the Game-Theory Scenarioscited by 3
  5. 2025Knowing What's Missing: Assessing Information Sufficiency in Question Answeringcited by 3
  6. 2026Cross-Lingual LLM-Judge Transfer via Evaluation Decompositioncited by 3
  7. 2025Are LLMs Good Text Diacritizers? An Arabic and Yoruba Case Studycited by 2
  8. 2025HybridEP: Scaling Expert Parallelism to Cross-Datacenter Scenario via Hybrid Expert/Data Transmissioncited by 2
  9. 2026ProMAS: Proactive Error Forecasting for Multi-Agent Systems Using Markov Transition Dynamicscited by 2
  10. 2026Tandem: Riding Together with Large and Small Language Models for Efficient Reasoningcited by 2
  11. 2025Can LLMs Solve ASP Problems? Insights from a Benchmarking Study (Extended Version)cited by 1
  12. 2025SHERPA: A Model-Driven Framework for Large Language Model Executioncited by 1
  13. 2025PerCache: Predictive Hierarchical Cache for RAG Applications on Mobile Devicescited by 1
  14. 2026dgMARK: Decoding-Guided Watermarking for Diffusion Language Modelscited by 1
  15. 2026Finding the Signal in the Noise: An Exploratory Study on Assessing the Effectiveness of AI and Accessibility Forums for Blind Users' Support Needscited by 1
  16. 2026ATOM: Instantiating Budget-Controllable Multi-Agent Collaboration via Nucleus-Electron Hierarchycited by 1
  17. 2026DeepDiscovery: A Location-Inference Framework for Task-Level Repository Understandingcited by 1
  18. 2026Intra-Prompt Parallel Decoding for Common-Context Question Answeringcited by 1
  19. 2025Rethinking Reasoning in LLMs: Neuro-Symbolic Local RetoMaton Beyond ICL and CoTcited by 0
  20. 2025ConvergeWriter: Data-Driven Bottom-Up Article Constructioncited by 0
  21. 2025Intelligence Foundation Model: A New Perspective to Approach Artificial General Intelligencecited by 0
  22. 2026How Is Uncertainty Propagated in Knowledge Distillation?cited by 0
  23. 2026Beyond Pipelines: A Fundamental Study on the Rise of Generative-Retrieval Architectures in Web Researchcited by 0
  24. 2026RouteGoT: Node-Adaptive Routing for Cost-Efficient Graph of Thoughts Reasoningcited by 0
  25. 2026LGMT: Logic-Grounded Metamorphic Testing for Evaluating the Reasoning Reliability of LLMscited by 0
  26. 2026Fighting Numerical Hallucinations via Data-centric Compilation for Online Financial QAcited by 0
  27. 2026HEART: Coordination of Heterogeneous Expert Agents for Physically Grounded Robotic Task Planningcited by 0
  28. 2026Benchmarking Large Language Models on Floating-Point Error Classificationcited by 0
  29. 2026TopoTuner: Topological Finetuning of Large Language Modelscited by 0
  30. 2026Asking Questions the Right Way: A Multi-Agent Conversational System for Prompt Formulation in Complex Task Resolutioncited by 0
  31. 2026VDGR-RAG: Vectors, Directories, Graphs, and Reflection Are All You Need for Unified Reasoning over Hierarchical Enterprise Knowledgecited by 0
  32. 2026From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injectioncited by 0
  33. 2026FOCUS & RePAIR: Mitigating Text Degeneration via Token-Level Guidance for Pruned Large Language Modelscited by 0
  34. 2026CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Agent-Based Detection and Resolution of Incompleteness and Ambiguity in Interactions with Large Language Models
  2. 2023On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answering
  3. 2026A Tri-Agent Framework for Evaluating and Aligning Question Clarification Capabilities of Large Language Models
  4. 2024Evaluating and Enhancing LLMs for Multi-turn Text-to-SQL with Multiple Question Types
  5. 2025MSCoRe: A Benchmark for Multi-Stage Collaborative Reasoning in LLM Agents
  6. 2024Multi-Agents Based on Large Language Models for Knowledge-based Visual Question Answering
  7. 2026MATA: Multi-Agent Framework for Reliable and Flexible Table Question Answering
  8. 2024Evaluating the Retrieval Component in LLM-Based Question Answering Systems