mlsanityData to 7 Oct 2026

ALFA: Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoning

arXiv 2502.14860 ↗First posted 20 Feb 2025cs.CLcited by 24

On the map: LLM reasoning → Clinical and biomedical applications

also in 1 more
Abstract (arXiv)

Large language models (LLMs) often fail to ask effective questions under uncertainty, making them unreliable in domains where proactive information-gathering is essential for decision-making. We present ALignment via Fine-grained Attributes, (ALFA) a framework that improves LLM question-asking by (i) decomposing the notion of a "good" question into a set of theory-grounded attributes (e.g., clarity, relevance), (ii) controllably synthesizing attribute-specific question variations, and (iii) aligning models via preference-based optimization to explicitly learn to ask better questions along these fine-grained attributes. Focusing on clinical reasoning as a case study, we introduce the MediQ-AskDocs dataset, composed of 17k real-world clinical interactions augmented with 80k attribute-specific preference pairs of follow-up questions, as well as a novel expert-annotated interactive healthcare QA task to evaluate question-asking abilities. Models aligned with ALFA reduce diagnostic errors by 56.6% on MediQ-AskDocs compared to SoTA instruction-tuned LLMs, with a question-level win-rate of 64.4% and strong generalizability. Our findings suggest that explicitly guiding question-asking with structured, fine-grained attributes offers a scalable path to improve LLMs, especially in expert application domains.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 36 of its 99 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

36 of 99 references.

References · 36

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  6. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  7. 2024Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Expertscited by 369
  8. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 299
  9. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 272
  10. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 271
  11. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 240
  12. 2024HelpSteer2: Open-source dataset for training top-performing reward modelscited by 232
24 more
  1. 2024WARM: On the Benefits of Weight Averaged Reward Modelscited by 145
  2. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
  3. 2023Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimizationcited by 121
  4. 2024STaR-GATE: Teaching Language Models to Ask Clarifying Questionscited by 99
  5. 2024Decoding-Time Language Model Alignment with Multiple Objectivescited by 90
  6. 2023AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Modelscited by 84
  7. 2023Clinical Camel: An Open Expert-Level Medical Language Model with Dialogue-Based Knowledge Encodingcited by 71
  8. 2024OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Frameworkcited by 70
  9. 2024Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questionscited by 70
  10. 2024Diverging Preferences: When do Annotators Disagree and do Models Know?cited by 47
  11. 2024Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Modelscited by 37
  12. 2024A Preliminary Study of o1 in Medicine: Are We Closer to an AI Doctor?cited by 33
  13. 2024Evaluating Language Models as Synthetic Data Generatorscited by 32
  14. 2024HealthQ: Unveiling Questioning Capabilities of LLM Chains in Healthcare Conversationscited by 31
  15. 2024MEDIC: Comprehensive Evaluation of Leading Indicators for LLM Safety and Utility in Clinical Applicationscited by 25
  16. 2024Open (Clinical) LLMs are Sensitive to Instruction Phrasingscited by 23
  17. 2025Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Mergingcited by 23
  18. 2024MCQG-SRefine: Multiple Choice Question Generation and Evaluation with Iterative Self-Critique, Correction, and Comparison Feedbackcited by 14
  19. 2022Privacy-Preserving Domain Adaptation of Semantic Parserscited by 12
  20. 2024SYNFAC-EDIT: Synthetic Imitation Edit Feedback for Factual Alignment in Clinical Summarizationcited by 12
  21. 2024ValueScope: Unveiling Implicit Norms and Values via Return Potential Model of Social Interactionscited by 12
  22. 2024Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domainscited by 12
  23. 2024The Limited Impact of Medical Adaptation of Large Language and Vision-Language Modelscited by 9
  24. 2024Generative AI in Medicinecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 15.

24 papers.

Sort:

By area:

  1. 2025Training Proactive and Personalized LLM Agentscited by 42
  2. 2025PrefPalette: Personalized Preference Modeling with Latent Attributescited by 15
  3. 2025TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Storiescited by 10
  4. 2025MedKGI: Iterative Differential Diagnosis with Medical Knowledge Graphs and Information-Guided Inquiringcited by 7
  5. 2026MultiSessionCollab: Learning User Preferences with Memory to Improve Long-Term Collaborationcited by 7
  6. 2025Beyond Passive Critical Thinking: Fostering Proactive Questioning to Enhance Human-AI Collaborationcited by 5
  7. 2026InfoGatherer: Principled Information Seeking via Evidence Retrieval and Strategic Questioningcited by 3
  8. 2026Beyond Idealized Patients: Evaluating LLMs under Challenging Patient Behaviors in Medical Consultationscited by 3
  9. 2026GoodPoint: Learning Constructive Scientific Paper Feedback from Author Responsescited by 3
  10. 2026Clarify or Answer: Reinforcement Learning for Agentic VQA with Context Under-specificationcited by 2
  11. 2026Cold-Start Personalization via Training-Free Priors from Structured World Modelscited by 2
  12. 2026MedConceal: A Benchmark for Clinical Hidden-Concern Reasoning Under Partial Observabilitycited by 2
12 more
  1. 2025BehaviorSFT: Behavioral Token Conditioning for Clinical Agents Across the Proactivity Spectrumcited by 1
  2. 2026Linking Knowledge to Care: Knowledge Graph-Augmented Medical Follow-Up Question Generationcited by 1
  3. 2025RACE-Align: Retrieval-Augmented and Chain-of-Thought Enhanced Preference Alignment for Large Language Modelscited by 0
  4. 2025Towards Better Health Conversations: The Benefits of Context-seekingcited by 0
  5. 2025Socratic Students: Teaching Language Models to Learn by Asking Questionscited by 0
  6. 2026How Much Would a Clinician Edit This Draft? Evaluating LLM Alignment for Patient Message Response Draftingcited by 0
  7. 2026CA-BED: Conversation-Aware Bayesian Experimental Designcited by 0
  8. 2026LLM-Guided Evolution for Medical Decision Pipelinescited by 0
  9. 2026PatientAgentBench: A Benchmark Framework for Evaluating Patient-Facing Health AI Agentscited by 0
  10. 2026Asking Questions the Right Way: A Multi-Agent Conversational System for Prompt Formulation in Complex Task Resolutioncited by 0
  11. 2026Guideline-as-Oracle: Zero-Annotation Training of an Ophthalmic Telephone Triage Agentcited by 0
  12. 2026PCQC: Privileged Counterfactual Question Credit for Multi-Turn Medical Dialoguecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning
  2. 2023Aligning Large Language Models for Clinical Tasks
  3. 2025Evaluating Large Language Models for Evidence-Based Clinical Question Answering
  4. 2024LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Models
  5. 2025Structured Outputs Enable General-Purpose LLMs to be Medical Experts
  6. 2025Collaboration among Multiple Large Language Models for Medical Question Answering
  7. 2026Yale-DM-Lab at ArchEHR-QA 2026: Deterministic Grounding and Multi-Pass Evidence Alignment for EHR Question Answering
  8. 2025Trustworthy Medical Question Answering: An Evaluation-Centric Survey