# Prompt-Robust Language Models: Which Training Strategies Work?

arXiv: https://arxiv.org/abs/2609.01217 · HTML: https://mlsanity.com/paper/2609.01217/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Builds on (most cited first, 20 of 20)

- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [State of What Art? A Call for Multi-Prompt LLM Evaluation](https://mlsanity.com/paper/2401.00595/) (2023)
- [ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs](https://mlsanity.com/paper/2410.12405/) (2024)
- [Does Prompt Formatting Have Any Impact on LLM Performance?](https://mlsanity.com/paper/2411.10541/) (2024)
- [Prompt Consistency for Zero-Shot Task Generalization](https://mlsanity.com/paper/2205.00049/) (2022)
- [POSIX: A Prompt Sensitivity Index For Large Language Models](https://mlsanity.com/paper/2410.02185/) (2024)
- [Prompt Perturbation Consistency Learning for Robust Language Models](https://mlsanity.com/paper/2402.15833/) (2024)
- [DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation](https://mlsanity.com/paper/2503.01622/) (2025)
- [Contrastive Instruction Tuning](https://mlsanity.com/paper/2402.11138/) (2024)
- [Enhancing LLM Robustness to Perturbed Instructions: An Empirical Study](https://mlsanity.com/paper/2504.02733/) (2025)
- [Unveiling the Lexical Sensitivity of LLMs: Combinatorial Optimization for Prompt Enhancement](https://mlsanity.com/paper/2405.20701/) (2024)
- [Forget What You Know about LLMs Evaluations -- LLMs are Like a Chameleon](https://mlsanity.com/paper/2502.07445/) (2025)
- [When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs](https://mlsanity.com/paper/2508.11383/) (2025)
- [Auto-Prompt Generation is Not Robust: Prompt Optimization Driven by Pseudo Gradient](https://mlsanity.com/paper/2412.18196/) (2024)
- [CREME: Robustness Enhancement of Code LLMs via Layer-Aware Model Editing](https://mlsanity.com/paper/2507.16407/) (2025)
- [Reinforcement Learning for Aligning Large Language Models Agents with Interactive Environments: Quantifying and Mitigating Prompt Overfitting](https://mlsanity.com/paper/2410.19920/) (2024)
- [Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs](https://mlsanity.com/paper/2510.14242/) (2025)
- [PAFT: Prompt-Agnostic Fine-Tuning](https://mlsanity.com/paper/2502.12859/) (2025)

## Abstract (arXiv)

Despite their strong performance, large language models remain highly sensitive to prompt formulation. Prior work addresses this through refined data construction or through dedicated robustness objectives. We reproduce and compare these strategies under controlled conditions, and measure how effective they are in addressing models' prompt sensitivity. We find the current robustness fine-tuning methods improve over standard fine-tuning and in-context learning, but the best-to-worst prompt gap remains as high as 40-57% of performance. Moreover, the recent robustness-enhancing methods we test - CoIN for contrastive alignment and PPCL for consistency regularization - often fail to outperform the simplest data construction strategy: training on one template per batch. Our diagnostics explain these results. The auxiliary objectives move the quantity they penalize, but do not generalize beyond it. Additionally, data construction strategies differ due to the conflicting signs of per-template gradients on 57-64% of parameters. Thus, batches that mix formulations force the optimizer to reconcile competing updates instead of finding a shared, prompt-agnostic one.