mlsanityData to 7 Oct 2026

DrugClaw and DrugAudit: A Primary-Source-Grounded Agent and Authority-Aware Benchmark for Drug-Information Question Answering

arXiv 2606.01434 ↗First posted 31 May 2026cs.CLcited by 0

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

Drug-information question answering is a high-stakes setting where hallucinated facts can mislead clinical decision-making and the provenance of each cited fact matters as much as the fact itself. We present DrugClaw, a multi-agent retrieval-augmented system that queries a registry of drug and pharmacovigilance skills via a reflection-driven state-machine workflow and returns answers grounded in primary regulatory or peer-reviewed records. We also contribute DrugAudit, a 3,772-item authority-aware benchmark with an evaluation panel that scores upstream-of-gold source match, token-level semantic snippet overlap, and citation faithfulness under a dual-judge LLM-as-judge protocol with inter-judge kappa = 0.88 (almost-perfect). Across DrugAudit plus drug-related subsets of MedQA (751) and PubMedQA (512), DrugClaw is top-1 on every column of the headline table: composite Evidence Index under both judges, judge-mediated answer correctness, primary-source rate (0.918, +10.1 pp over next-best), faithfulness (0.887, +5.9 pp), MedQA (0.920), and PubMedQA (0.693).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 30 of its 58 references that are in the map. First posted on arXiv 31 May 2026.

Builds on

30 of 58 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025TxAgent: An AI Agent for Therapeutic Reasoning Across a Universe of Tools14 Mar

Other references · 29

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  6. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  7. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  8. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  9. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  10. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  11. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  12. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
17 more
  1. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  2. 2022Galactica: A Large Language Model for Sciencecited by 736
  3. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
  4. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  5. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
  6. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  7. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  8. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  9. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 202
  10. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 146
  11. 2024Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluatorscited by 146
  12. 2023Almanac: Retrieval-Augmented Language Models for Clinical Medicinecited by 122
  13. 2024Empowering Biomedical Discovery with AI Agentscited by 117
  14. 2024MedAdapter: Efficient Test-Time Adaptation of Large Language Models towards Medical Reasoningcited by 35
  15. 2025Think-on-Graph 3.0: Efficient and Adaptive LLM Reasoning on Heterogeneous Graphs via Multi-Agent Dual-Evolving Context Retrievalcited by 10
  16. 2024DrugWatch: A Comprehensive Multi-Source Data Visualisation Platform for Drug Safety Informationcited by 2
  17. 2025DeepEvidence: Empowering Biomedical Discovery with Deep Knowledge Graph Researchcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FDARxBench: Benchmarking Regulatory and Clinical Reasoning on FDA Generic Drug Assessment
  2. 2026PubMed Reasoner: Dynamic Reasoning-based Retrieval for Evidence-Grounded Biomedical Question Answering
  3. 2025Agentic Medical Knowledge Graphs Enhance Medical Question Answering: Bridging the Gap Between LLMs and Evolving Medical Knowledge
  4. 2025DrugRAG: Enhancing Pharmacy LLM Performance Through A Novel Retrieval-Augmented Generation Pipeline
  5. 2026LakeQuest: A Three-Domain Benchmark for Grounded Question Answering across Data Lakes
  6. 2026ClinAgent: A ReAct-Based Agent for Conversational Access to Clinical Trial Information
  7. 2025MedTrust-RAG: Evidence Verification and Trust Alignment for Biomedical Question Answering
  8. 2026UIC-AIHealth4All at ArchEHR-QA 2026: Answer-First Evidence Grounding for Clinical Question Answering