# PVminerLLM2: Improving Structured Extraction of Patient Voice via Preference Optimization

arXiv: https://arxiv.org/abs/2606.16074 · HTML: https://mlsanity.com/paper/2606.16074/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Medical LLM alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 15 Jun 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Medical LLM alignment](https://mlsanity.com/alignment/medical-llm-alignment/)

## Builds on (most cited first, 16 of 16)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [SimPO: Simple Preference Optimization with a Reference-Free Reward](https://mlsanity.com/paper/2405.14734/) (2024)
- [A General Theoretical Paradigm to Understand Learning from Human Preferences](https://mlsanity.com/paper/2310.12036/) (2023)
- [Token-level Direct Preference Optimization](https://mlsanity.com/paper/2404.11999/) (2024)
- [Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignment](https://mlsanity.com/paper/2412.14516/) (2024)
- [MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration](https://mlsanity.com/paper/2506.19835/) (2025)
- [PlanGEN: A Multi-Agent Framework for Generating Planning and Reasoning Trajectories for Complex Problem Solving](https://mlsanity.com/paper/2502.16111/) (2025)
- [Token-Importance Guided Direct Preference Optimization](https://mlsanity.com/paper/2505.19653/) (2025)
- [TAB-PO: Preference Optimization with a Token-Level Adaptive Barrier for Token-Critical Structured Generation](https://mlsanity.com/paper/2603.00025/) (2026)
- [EPPCMinerBen: A Novel Benchmark for Evaluating Large Language Models on Electronic Patient-Provider Communication via the Patient Portal](https://mlsanity.com/paper/2603.00028/) (2026)
- [PVminer: A Domain-Specific Tool to Detect the Patient Voice in Patient Generated Data](https://mlsanity.com/paper/2602.21165/) (2026)
- [PVminerLLM: Structured Extraction of Patient Voice from Patient-Generated Text using Large Language Models](https://mlsanity.com/paper/2603.05776/) (2026)
- [STaR-DRO: Stateful Tsallis Reweighting for Group-Robust Structured Prediction](https://mlsanity.com/paper/2604.09737/) (2026)
- [EPPC-OASIS: Ontology-Aware Adaptation and Structured Inference Refinement for Electronic Patient-Provider Communication Mining in Secure Messages](https://mlsanity.com/paper/2605.24172/) (2026)

## Abstract (arXiv)

Motivation: Patient-generated text contains critical information on patients' lived experiences, social context, and care engagement, but remains largely unstructured, limiting its use in patient-centered outcomes research. Prior work introduced the PV-Miner benchmark and PVMinerLLM models for structured extraction. However, supervised fine-tuning (SFT) alone struggles with rare, fine-grained, and unevenly distributed errors, particularly in token-critical structured outputs. Results: We present PVminerLLM2, an improved set of LLMs for structured patient voice extraction that applies preference optimization to address token-critical errors beyond the reach of supervised fine-tuning. Our method introduces (i) a preference objective with token-level gated stabilization term that prevents degradation of absolute token likelihood under preference optimization, and (ii) confusion-aware preference pair construction to better capture low-separation distinctions. We further incorporate token-importance weighting and inverse-frequency reweighing to address token imbalance and class skew. Across multiple model sizes, PVMinerLLM2 consistently outperforms strong baselines, achieving gains of up to 4.43% (Code), 3.50% (Sub-code), and 1.55% (Span), and outperforms baseline LLM trained with existing preference optimization methods. Availability and Implementation: The supplementary material, code, evaluation scripts, and trained models for PVminerLLM2 are publicly available at: https://github.com/Data-Mining-Lab-Yale/PVminerLLM2