mlsanityData to 14 Oct 2026

PVminerLLM2: Improving Structured Extraction of Patient Voice via Preference Optimization

arXiv 2606.16074 ↗First posted 15 Jun 2026cs.CL cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Medical LLM alignment

Abstract (arXiv)

Motivation: Patient-generated text contains critical information on patients' lived experiences, social context, and care engagement, but remains largely unstructured, limiting its use in patient-centered outcomes research. Prior work introduced the PV-Miner benchmark and PVMinerLLM models for structured extraction. However, supervised fine-tuning (SFT) alone struggles with rare, fine-grained, and unevenly distributed errors, particularly in token-critical structured outputs. Results: We present PVminerLLM2, an improved set of LLMs for structured patient voice extraction that applies preference optimization to address token-critical errors beyond the reach of supervised fine-tuning. Our method introduces (i) a preference objective with token-level gated stabilization term that prevents degradation of absolute token likelihood under preference optimization, and (ii) confusion-aware preference pair construction to better capture low-separation distinctions. We further incorporate token-importance weighting and inverse-frequency reweighing to address token imbalance and class skew. Across multiple model sizes, PVMinerLLM2 consistently outperforms strong baselines, achieving gains of up to 4.43% (Code), 3.50% (Sub-code), and 1.55% (Span), and outperforms baseline LLM trained with existing preference optimization methods. Availability and Implementation: The supplementary material, code, evaluation scripts, and trained models for PVminerLLM2 are publicly available at: https://github.com/Data-Mining-Lab-Yale/PVminerLLM2

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical LLM alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 16 of its 27 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

16 of 27 references.

References · 16

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  3. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
  4. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  5. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
  6. 2024Token-level Direct Preference Optimizationcited by 135
  7. 2024Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignmentcited by 51
  8. 2025MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaborationcited by 40
  9. 2025PlanGEN: A Multi-Agent Framework for Generating Planning and Reasoning Trajectories for Complex Problem Solvingcited by 15
  10. 2025Token-Importance Guided Direct Preference Optimizationcited by 14
  11. 2026TAB-PO: Preference Optimization with a Token-Level Adaptive Barrier for Token-Critical Structured Generationcited by 7
  12. 2026EPPCMinerBen: A Novel Benchmark for Evaluating Large Language Models on Electronic Patient-Provider Communication via the Patient Portalcited by 4
4 more
  1. 2026PVminer: A Domain-Specific Tool to Detect the Patient Voice in Patient Generated Datacited by 3
  2. 2026PVminerLLM: Structured Extraction of Patient Voice from Patient-Generated Text using Large Language Modelscited by 2
  3. 2026STaR-DRO: Stateful Tsallis Reweighting for Group-Robust Structured Predictioncited by 2
  4. 2026EPPC-OASIS: Ontology-Aware Adaptation and Structured Inference Refinement for Electronic Patient-Provider Communication Mining in Secure Messagescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Empathy by Design: Aligning Large Language Models for Healthcare Dialogue
  2. 2025Empowering Healthcare Practitioners with Language Models: Structuring Speech Transcripts in Two Real-World Clinical Applications
  3. 2026DiaWhisper-DPO: Role-Attributed Transcription of Clinical Interviews via Failure-Mined Preference Optimization
  4. 2025Conversation AI Dialog for Medicare powered by Finetuning and Retrieval Augmented Generation
  5. 2025IIMedGPT: Promoting Large Language Model Capabilities of Medical Tasks by Efficient Human Preference Alignment
  6. 2026Retrieval-Augmented Large Language Models for Schema-Constrained Clinical Information Extraction
  7. 2025Leveraging Language Models for Automated Patient Record Linkage
  8. 2025Context-Aware Classification and Grading of Sensitive Information in Online Conversational Health Data