mlsanityData to 7 Oct 2026

Dr.Copilot: A Multi-Agent Prompt Optimized Assistant for Improving Patient-Doctor Communication in Romanian

arXiv 2507.11299 ↗First posted 15 Jul 2025cs.CLcited by 4

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

Text-based telemedicine has become increasingly common, yet the quality of medical advice in doctor-patient interactions is often judged more on how advice is communicated rather than its clinical accuracy. To address this, we introduce Dr. Copilot , a multi-agent large language model (LLM) system that supports Romanian-speaking doctors by evaluating and enhancing the presentation quality of their written responses. Rather than assessing medical correctness, Dr. Copilot provides feedback along 17 interpretable axes. The system comprises of three LLM agents with prompts automatically optimized via DSPy. Designed with low-resource Romanian data and deployed using open-weight models, it delivers real-time specific feedback to doctors within a telemedicine platform. Empirical evaluations and live deployment with 41 doctors show measurable improvements in user reviews and response quality, marking one of the first real-world deployments of LLMs in Romanian medical settings.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 23 of its 52 references that are in the map. First posted on arXiv 15 Jul 2025.

Builds on

23 of 52 references.

References · 23

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  4. 2024A Survey on LLM-as-a-Judgecited by 1,384
  5. 2022Red Teaming Language Models with Language Modelscited by 983
  6. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  7. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  8. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  9. 2020Towards a Human-like Open-Domain Chatbotcited by 467
  10. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  11. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 354
  12. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
11 more
  1. 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 324
  2. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 282
  3. 2024Self-Preference Bias in LLM-as-a-Judgecited by 248
  4. 2020Unsupervised Evaluation of Interactive Dialog with DialoGPTcited by 139
  5. 2023Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversationscited by 88
  6. 2025MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilotcited by 83
  7. 2023A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluatorscited by 45
  8. 2019Generating Empathetic Responses by Looking Ahead the User's Sentimentcited by 27
  9. 2024Polaris: A Safety-focused LLM Constellation Architecture for Healthcarecited by 27
  10. 2018Topic-based Evaluation for Conversational Botscited by 16
  11. 2021Should we Stop Training More Monolingual Models, and Simply Use Machine Translation Instead?cited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Automatic Prompt Optimization for Dataset-Level Feature Discoverycited by 2
  2. 2026Challenges and Recommendations for LLM-as-a-Judge in Multilingual Settings and for Low-Resource Languagescited by 1
  3. 2026What Makes a Good Doctor Response? A Study on Text-Based Telemedicinecited by 0
  4. 2026Improving Medical Communication using Rubric-Guided Counterfactual Recommendationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025AutoMedic: An Automated Evaluation Framework for Clinical Conversational Agents with Medical Dataset Grounding
  2. 2025Enabling Doctor-Centric Medical AI with LLMs through Workflow-Aligned Tasks and Benchmarks
  3. 2023Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review
  4. 2026MedDialogRubrics: A Comprehensive Benchmark and Evaluation Framework for Multi-turn Medical Consultations in Large Language Models
  5. 2024Agentic LLM Workflows for Generating Patient-Friendly Medical Reports
  6. 2025OpenAIs HealthBench in Action: Evaluating an LLM-Based Medical Assistant on Realistic Clinical Queries
  7. 2025Architecting Clinical Collaboration: Multi-Agent Reasoning Systems for Multimodal Medical VQA
  8. 2025IMB: An Italian Medical Benchmark for Question Answering