mlsanityData to 14 Oct 2026

DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents

arXiv 2303.17071 ↗First posted 30 Mar 2023cs.CL cs.AI cs.LGcited by 52

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

Large language models (LLMs) have emerged as valuable tools for many natural language understanding tasks. In safety-critical applications such as healthcare, the utility of these models is governed by their ability to generate outputs that are factually accurate and complete. In this work, we present dialog-enabled resolving agents (DERA). DERA is a paradigm made possible by the increased conversational abilities of LLMs, namely GPT-4. It provides a simple, interpretable forum for models to communicate feedback and iteratively improve output. We frame our dialog as a discussion between two agent types - a Researcher, who processes information and identifies crucial problem components, and a Decider, who has the autonomy to integrate the Researcher's information and makes judgments on the final output. We test DERA against three clinically-focused tasks. For medical conversation summarization and care plan generation, DERA shows significant improvement over the base GPT-4 performance in both human expert preference evaluations and quantitative metrics. In a new finding, we also show that GPT-4's performance (70%) on an open-ended version of the MedQA question-answering (QA) dataset (Jin et al. 2021, USMLE) is well above the passing level (60%), with DERA showing similar performance. We release the open-ended MEDQA dataset at https://github.com/curai/curai-research/tree/main/DERA.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 26 of its 79 references that are in the map. First posted on arXiv 30 Mar 2023.

Builds on

26 of 79 references.

References · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  8. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  9. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  10. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  11. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  12. 2022Large Language Models Can Self-Improvecited by 664
14 more
  1. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  2. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  3. 2022News Summarization and Evaluation in the Era of GPT-3cited by 327
  4. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 268
  5. 2022Can large language models reason about medical questions?cited by 194
  6. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 138
  7. 2023ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptionscited by 86
  8. 2021Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarizationcited by 76
  9. 2022Iteratively Prompt Pre-trained Language Models for Chain of Thoughtcited by 75
  10. 2020Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structurescited by 49
  11. 2022BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generationcited by 48
  12. 2021Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversationscited by 42
  13. 2022Entailer: Answering Questions with Faithful and Truthful Chains of Reasoningcited by 38
  14. 2016On the Diagnostic of Road Pathway Visibilitycited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 26; 2024: 16; 2025: 8; 2026: 2.

52 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  3. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  4. 2023Teaching Large Language Models to Self-Debugcited by 988
  5. 2023Large Language Models as Optimizerscited by 864
  6. 2024Large Language Models: A Surveycited by 525
  7. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 384
  8. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 260
  9. 2023RoCo: Dialectic Multi-Robot Collaboration with Large Language Modelscited by 172
  10. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 127
  11. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  12. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agentscited by 125
38 more
  1. 2023Deception Abilities Emerged in Large Language Modelscited by 101
  2. 2023AgentSims: An Open-Source Sandbox for Large Language Model Evaluationcited by 88
  3. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 82
  4. 2023Jailbreaking GPT-4V via Self-Adversarial Attacks with System Promptscited by 65
  5. 2026Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontierscited by 65
  6. 2023Emotional Intelligence of Large Language Modelscited by 58
  7. 2023CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translationcited by 57
  8. 2023MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Recordscited by 56
  9. 2023Stance Detection with Collaborative Role-Infused LLM-Based Agentscited by 53
  10. 2024Autonomous LLM-driven research from data to human-verifiable research paperscited by 53
  11. 2024Prompt Design and Engineering: Introduction and Advanced Methodscited by 45
  12. 2023CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Updatecited by 43
  13. 2023Video ChatCaptioner: Towards Enriched Spatiotemporal Descriptionscited by 42
  14. 2023CGMI: Configurable General Multi-Agent Interaction Frameworkcited by 42
  15. 2024CYCLE: Learning to Self-Refine the Code Generationcited by 42
  16. 2023chatClimate: Grounding Conversational AI in Climate Sciencecited by 32
  17. 2024RL-GPT: Integrating Reinforcement Learning and Code-as-policycited by 32
  18. 2024Materials science in the era of large language models: a perspectivecited by 24
  19. 2023Learning by Self-Explainingcited by 17
  20. 2023Designing LLM Chains by Adapting Techniques from Crowdsourcing Workflowscited by 17
  21. 2023InterAct: Exploring the Potentials of ChatGPT as a Cooperative Agentcited by 16
  22. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 15
  23. 2023Evaluation of large language models for discovery of gene set functioncited by 14
  24. 2024An Electoral Approach to Diversify LLM-based Multi-Agent Collective Decision-Makingcited by 14
  25. 2024From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Modelscited by 8
  26. 2023Self-Convinced Prompting: Few-Shot Question Answering with Repeated Introspectioncited by 7
  27. 2024MCRanker: Generating Diverse Criteria On-the-Fly to Improve Point-wise LLM Rankerscited by 7
  28. 2025Exploiting Edited Large Language Models as General Scientific Optimizerscited by 7
  29. 2023Extrinsically-Focused Evaluation of Omissions in Medical Summarizationcited by 6
  30. 2023CONSCENDI: A Contrastive and Scenario-Guided Distillation Approach to Guardrail Models for Virtual Assistantscited by 5
  31. 2024Co-Learning: Code Learning for Multi-Agent Reinforcement Collaborative Framework with Conversational Natural Language Interfacescited by 5
  32. 2024LLMR: Knowledge Distillation with a Large Language Model-Induced Rewardcited by 3
  33. 2025MADP: Multi-Agent Deductive Planning for Enhanced Cognitive-Behavioral Mental Health Question Answercited by 3
  34. 2025Nature's Insight: A Novel Framework and Comprehensive Analysis of Agentic Reasoning Through the Lens of Neurosciencecited by 3
  35. 2023Human-Like Intuitive Behavior and Reasoning Biases Emerged in Language Models -- and Disappeared in GPT-4cited by 2
  36. 2025Rare Disease Differential Diagnosis with Large Language Models at Scale: From Abdominal Actinomycosis to Wilson's Diseasecited by 2
  37. 2024Linguacodus: A Synergistic Framework for Transformative Code Generation in Machine Learning Pipelinescited by 0
  38. 2025Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodingscited by 0

50 most cited of 52 All 52 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023DiagGPT: An LLM-based and Multi-agent Dialogue System with Automatic Topic Management for Flexible Task-Oriented Dialogue
  2. 2025AutoMedic: An Automated Evaluation Framework for Clinical Conversational Agents with Medical Dataset Grounding
  3. 2023Challenges of GPT-3-based Conversational Agents for Healthcare
  4. 2025Conversation AI Dialog for Medicare powered by Finetuning and Retrieval Augmented Generation
  5. 2026MedDialogRubrics: A Comprehensive Benchmark and Evaluation Framework for Multi-turn Medical Consultations in Large Language Models
  6. 2023Conversational Health Agents: A Personalized LLM-Powered Agent Framework
  7. 2024Tool Calling: Enhancing Medication Consultation via Retrieval-Augmented Large Language Models
  8. 2025Dialogue is Better Than Monologue: Instructing Medical LLMs via Strategical Conversations