# BitHydra: Towards Bit-flip Inference Cost Attack against Large Language Models

arXiv: https://arxiv.org/abs/2505.16670 · HTML: https://mlsanity.com/paper/2505.16670/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 79 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://mlsanity.com/paper/2405.04434/) (2024)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [TBT: Targeted Neural Network Attack with Bit Trojan](https://mlsanity.com/paper/1909.05193/) (2019)
- [Sponge Examples: Energy-Latency Attacks on Neural Networks](https://mlsanity.com/paper/2006.03463/) (2020)
- [Bit-Flip Attack: Crushing Neural Network with Progressive Bit Search](https://mlsanity.com/paper/1903.12269/) (2019)
- [OverThink: Slowdown Attacks on Reasoning LLMs](https://mlsanity.com/paper/2502.02542/) (2025)
- [Coercing LLMs to do and reveal (almost) anything](https://mlsanity.com/paper/2402.14020/) (2024)
- [Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images](https://mlsanity.com/paper/2401.11170/) (2024)
- [A Panda? No, It's a Sloth: Slowdown Attacks on Adaptive Multi-Exit Neural Network Inference](https://mlsanity.com/paper/2010.02432/) (2020)
- [DeepHammer: Depleting the Intelligence of Deep Neural Networks through Targeted Chain of Bit Flips](https://mlsanity.com/paper/2003.13746/) (2020)
- [NICGSlowDown: Evaluating the Efficiency Robustness of Neural Image Caption Generation Models](https://mlsanity.com/paper/2203.15859/) (2022)
- [Targeted Attack against Deep Neural Networks via Flipping Limited Weight Bits](https://mlsanity.com/paper/2102.10496/) (2021)
- [DeepSteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memories](https://mlsanity.com/paper/2111.04625/) (2021)
- [In ChatGPT We Trust? Measuring and Characterizing the Reliability of ChatGPT](https://mlsanity.com/paper/2304.08979/) (2023)
- [An Engorgio Prompt Makes Large Language Model Babble on](https://mlsanity.com/paper/2412.19394/) (2024)
- [Phantom Sponges: Exploiting Non-Maximum Suppression to Attack Deep Object Detectors](https://mlsanity.com/paper/2205.13618/) (2022)
- [RowHammer: A Retrospective](https://mlsanity.com/paper/1904.09724/) (2019)
- [RADAR: Run-time Adversarial Weight Attack Detection and Accuracy Recovery](https://mlsanity.com/paper/2101.08254/) (2021)
- [Aegis: Mitigating Targeted Bit-flip Attacks against Deep Neural Networks](https://mlsanity.com/paper/2302.13520/) (2023)
- [LLMEffiChecker: Understanding and Testing Efficiency Degradation of Large Language Models](https://mlsanity.com/paper/2210.03696/) (2022)
- [SlowTrack: Increasing the Latency of Camera-based Perception in Autonomous Driving Using Adversarial Examples](https://mlsanity.com/paper/2312.09520/) (2023)
- [One-bit Flip is All You Need: When Bit-flip Attack Meets Model Training](https://mlsanity.com/paper/2308.07934/) (2023)

## Built on by (most cited first, 2 of 2)

- [BenchOverflow: Measuring Overflow in Large Language Models via Plain-Text Prompts](https://mlsanity.com/paper/2601.08490/) (2026)
- [Rethinking Latency Denial-of-Service: Attacking the LLM Serving Framework, Not the Model](https://mlsanity.com/paper/2602.07878/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are widely deployed, but their substantial compute demands make them vulnerable to inference cost attacks that aim to deliberately maximize the output length. In this work, we investigate a distinct attack surface: maximizing inference cost by tampering with the model parameters instead of inputs. This approach leverages the established capability of Bit-Flip Attacks (BFAs) to persistently alter model behavior via minute weight perturbations, effectively decoupling the attack from specific input queries. To realize this, we propose BitHydra, a framework that addresses the unique optimization challenge of identifying the exact weight bits that maximize generation cost. We formulate the attack as a constrained Binary Integer Programming (BIP) problem designed to systematically suppress the end-of-sequence (i.e., <eos>) probability. To overcome the intractability of the discrete search space, we relax the problem into a continuous optimization task and solve it via the Alternating Direction Method of Multipliers (ADMM). We evaluate BitHydra across 10 LLMs (1.5B-16B). Our results demonstrate that the proposed optimization method efficiently achieves endless generation with as few as 1-4 bit flips on all testing models, verifying the effectiveness of the ADMM-based formulation against both standard models and potential defenses.