# BenchOverflow: Measuring Overflow in Large Language Models via Plain-Text Prompts

arXiv: https://arxiv.org/abs/2601.08490 · HTML: https://mlsanity.com/paper/2601.08490/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- Builds on 35 of its 46 references that are in the map.
- First posted on arXiv 13 Jan 2026.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 30 of 35)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [Open Sesame! Universal Black Box Jailbreaking of Large Language Models](https://mlsanity.com/paper/2309.01446/) (2023)
- [OverThink: Slowdown Attacks on Reasoning LLMs](https://mlsanity.com/paper/2502.02542/) (2025)
- [Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification](https://mlsanity.com/paper/2407.20859/) (2024)
- [Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images](https://mlsanity.com/paper/2401.11170/) (2024)
- [Denial-of-Service Poisoning Attacks against Large Language Models](https://mlsanity.com/paper/2410.10760/) (2024)
- [Crabs: Consuming Resource via Auto-generation for LLM-DoS Attack under Black-box Settings](https://mlsanity.com/paper/2412.13879/) (2024)
- [An Engorgio Prompt Makes Large Language Model Babble on](https://mlsanity.com/paper/2412.19394/) (2024)
- [Traceback of Poisoning Attacks to Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.21668/) (2025)
- [Excessive Reasoning Attack on Reasoning LLMs](https://mlsanity.com/paper/2506.14374/) (2025)
- [POT: Inducing Overthinking in LLMs via Black-Box Iterative Optimization](https://mlsanity.com/paper/2508.19277/) (2025)
- [ExtendAttack: Attacking Servers of LRMs via Extending Reasoning](https://mlsanity.com/paper/2506.13737/) (2025)

## Built on by (most cited first, 3 of 3)

- [Rethinking Latency Denial-of-Service: Attacking the LLM Serving Framework, Not the Model](https://mlsanity.com/paper/2602.07878/) (2026)
- [Resource Consumption Threats in Large Language Models](https://mlsanity.com/paper/2603.16068/) (2026)
- [Talk is (Not) Cheap: A Taxonomy and Benchmark Coverage Audit for LLM Attacks](https://mlsanity.com/paper/2605.15118/) (2026)

## Abstract (arXiv)

We investigate a failure mode of large language models (LLMs) in which plain-text prompts elicit excessive outputs, a phenomenon we term Overflow. Unlike jailbreaks or prompt injection, Overflow arises under ordinary interaction settings and can lead to elevated serving cost, latency, and cross-user performance degradation, particularly when scaled across many requests. Beyond usability, the stakes are economic and environmental: unnecessary tokens increase per-request cost and energy consumption, compounding into substantial operational spend and carbon footprint at scale. Moreover, Overflow represents a practical vector for compute amplification and service degradation in shared environments. We introduce BenchOverflow, a model-agnostic benchmark of nine plain-text prompting strategies that amplify output volume without adversarial suffixes or policy circumvention. Using a standardized protocol with a fixed budget of 5000 new tokens, we evaluate nine open- and closed-source models and observe pronounced rightward shifts and heavy tails in length distributions. Cap-saturation rates (CSR@1k/3k/5k) and empirical cumulative distribution functions (ECDFs) quantify tail risk; within-prompt variance and cross-model correlations show that Overflow is broadly reproducible yet heterogeneous across families and attack vectors. A lightweight mitigation-a fixed conciseness reminder-attenuates right tails and lowers CSR for all strategies across the majority of models. Our findings position length control as a measurable reliability, cost, and sustainability concern rather than a stylistic quirk. By enabling standardized comparison of length-control robustness across models, BenchOverflow provides a practical basis for selecting deployments that minimize resource waste and operating expense, and for evaluating defenses that curb compute amplification without eroding task performance.