# Stability-Aware Prompt Optimization for Clinical Data Abstraction

arXiv: https://arxiv.org/abs/2601.22373 · HTML: https://mlsanity.com/paper/2601.22373/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 16 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Builds on (most cited first, 7 of 7)

- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs](https://mlsanity.com/paper/2410.12405/) (2024)
- [Prompt Stability Matters: Evaluating and Optimizing Auto-Generated Prompt in General-Purpose Systems](https://mlsanity.com/paper/2505.13546/) (2025)
- [Survival of the Safest: Towards Secure Prompt Optimization through Interleaved Multi-Objective Evolution](https://mlsanity.com/paper/2410.09652/) (2024)
- [BlackDAN: A Black-Box Multi-Objective Approach for Effective and Contextual Jailbreaking of Large Language Models](https://mlsanity.com/paper/2410.09804/) (2024)
- [AEGIS : Automated Co-Evolutionary Framework for Guarding Prompt Injections Schema](https://mlsanity.com/paper/2509.00088/) (2025)
- [DRO-InstructZero: Distributionally Robust Prompt Optimization for Large Language Models](https://mlsanity.com/paper/2510.15260/) (2025)

## Abstract (arXiv)

Large language models used for clinical abstraction are sensitive to prompt wording, yet most work treats prompts as fixed and studies uncertainty in isolation. We argue these should be treated jointly. Across two clinical tasks (MedAlign applicability/correctness and MS subtype abstraction) and multiple open and proprietary models, we measure prompt sensitivity via flip rates and relate it to calibration and selective prediction. We find that higher accuracy does not guarantee prompt stability, and that models can appear well-calibrated yet remain fragile to paraphrases. We propose a dual-objective prompt optimization loop that jointly targets accuracy and stability, showing that explicitly including a stability term reduces flip rates across tasks and models, sometimes at modest accuracy cost. Our results suggest prompt sensitivity should be an explicit objective when validating clinical LLM systems.