# Hardware Mechanisms to Dynamically Throttle AI Performance

arXiv: https://arxiv.org/abs/2607.18069 · HTML: https://mlsanity.com/paper/2607.18069/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 70 references that are in the map.
- First posted on arXiv 20 Jul 2026.

## Builds on (most cited first, 18 of 18)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision](https://mlsanity.com/paper/2407.08608/) (2024)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [Thousands of AI Authors on the Future of AI](https://mlsanity.com/paper/2401.02843/) (2024)
- [Computing Power and the Governance of Artificial Intelligence](https://mlsanity.com/paper/2402.08797/) (2024)
- [Think Fast: Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models](https://mlsanity.com/paper/2606.07157/) (2026)
- [Hardware-Enabled Mechanisms for Verifying Responsible AI Development](https://mlsanity.com/paper/2505.03742/) (2025)
- [A Survey and Measurement Study of GPU DVFS on Energy Conservation](https://mlsanity.com/paper/1610.01784/) (2016)
- [The Loss of Control Playbook: Degrees, Dynamics, and Preparedness](https://mlsanity.com/paper/2511.15846/) (2025)
- [Vortex: Extending the RISC-V ISA for GPGPU and 3D-GraphicsResearch](https://mlsanity.com/paper/2110.10857/) (2021)
- [Hardware-Level Governance of AI Compute: A Feasibility Taxonomy for Regulatory Compliance and Treaty Verification](https://mlsanity.com/paper/2604.04712/) (2026)
- [Near-Term Enforcement of AI Chip Export Controls Using A Firmware-Based Design for Offline Licensing](https://mlsanity.com/paper/2404.18308/) (2024)
- [Embedded Off-Switches for AI Compute](https://mlsanity.com/paper/2509.07637/) (2025)
- [Latch-Based Logic Locking](https://mlsanity.com/paper/2005.10649/) (2020)

## Abstract (arXiv)

As more capable AI models are increasingly integrated into critical computer systems, the lack of control over AI intent motivates safety mechanisms. Existing software safeguards impose only behavioral constraints that can potentially be bypassed by sufficiently intelligent models. While hardware-level safety enforcement has been recognized as an essential last line of defense, few mechanisms have been proposed beyond policy regulations on unauthorized accesses or coarse full-chip shutdown. What is missing is a fine-grained, dynamic intervention mechanism at the architecture level. In this paper, we introduce a set of microarchitecture knobs which dynamically control the available hardware resources to limit AI performance at runtime. We evaluate candidate knobs spanning the GPU memory subsystem, across capacity, bandwidth, latency and frequency dimensions, and narrow down to four strong candidates: L2 size, L2 latency, L2 bandwidth, and shared memory port access rate. To minimize new logic and extra design cost, we build all four mechanisms from well-established microarchitectural primitives: cache way masking, credit-based rate limiting, latency insertion, and bank arbitration. We show that these knobs achieve high performance sensitivity (up to 80% performance cut at 1/8 resource availability), negligible implementation cost (<~10K flip flops), fast stabilization after dynamic throttling (5-80K cycles), and minimal collateral impact on the rest of the chip. Further, multi-knob analysis reveals combinations of knobs that amplify the performance degradation beyond the effect of each knob individually, which enables a broader range of performance targets.