# Evaluating Reliability Gaps in Large Language Model Safety via Repeated Prompt Sampling

arXiv: https://arxiv.org/abs/2604.09606 · HTML: https://mlsanity.com/paper/2604.09606/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 18 references that are in the map.
- First posted on arXiv 10 Mar 2026.

## Builds on (most cited first, 10 of 10)

- [Agent-SafetyBench: Evaluating the Safety of LLM Agents](https://mlsanity.com/paper/2412.14470/) (2024)
- [AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies](https://mlsanity.com/paper/2407.17436/) (2024)
- [SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types](https://mlsanity.com/paper/2410.21965/) (2024)
- [Calibrating LLM Confidence by Probing Perturbed Representation Stability](https://mlsanity.com/paper/2505.21772/) (2025)
- [Probabilistic Consensus through Ensemble Validation: A Framework for LLM Reliability](https://mlsanity.com/paper/2411.06535/) (2024)
- [Quantifying perturbation impacts for large language models](https://mlsanity.com/paper/2412.00868/) (2024)
- [Reducing Large Language Model Safety Risks in Women's Health using Semantic Entropy](https://mlsanity.com/paper/2503.00269/) (2025)
- [aiXamine: Simplified LLM Safety and Security](https://mlsanity.com/paper/2504.14985/) (2025)
- [The Instability of Safety: How Random Seeds and Temperature Expose Inconsistent LLM Refusal Behavior](https://mlsanity.com/paper/2512.12066/) (2025)
- [A Hierarchical Imprecise Probability Approach to Reliability Assessment of Large Language Models](https://mlsanity.com/paper/2511.00527/) (2025)

## Built on by (most cited first, 1 of 1)

- [Prioritizing Repeated LLM Evaluation for Hidden Failure Discovery](https://mlsanity.com/paper/2609.32547/) (2026)

## Abstract (arXiv)

Traditional benchmarks for large language models (LLMs), such as HELM and AIR-BENCH, primarily assess safety risk through breadth-oriented evaluation across diverse tasks. However, real-world deployment often exposes a different class of risk: operational failures arising from repeated generations of the same prompt rather than broad task generalization. In high-stakes settings, response consistency and safety under repeated use are critical operational requirements. We introduce Accelerated Prompt Stress Testing (APST), a depth-oriented evaluation framework inspired by highly accelerated stress testing in reliability engineering. APST probes LLM behavior by repeatedly sampling identical prompts under controlled operational conditions, including temperature variation and prompt perturbation, to surface latent failure modes such as hallucinations, refusal inconsistency, and unsafe completions. Rather than treating failures as isolated events, APST characterizes them statistically as stochastic outcomes of repeated inference. We model observed safety failures using Bernoulli and binomial formulations to estimate per-inference failure probabilities, enabling quantitative comparison of operational risk across models and configurations. We apply APST to multiple instruction-tuned LLMs evaluated on AIR-BENCH 2024 derived safety and security prompts. While models exhibit similar performance under conventional single- or very-low-sample evaluation (N <= 3), repeated sampling reveals substantial variation in empirical failure probabilities across temperatures. These results demonstrate that shallow benchmark scores can obscure meaningful differences in reliability under sustained use.