# LIMINAL: Exploring The Frontiers of LLM Decode Performance

arXiv: https://arxiv.org/abs/2507.14397 · HTML: https://mlsanity.com/paper/2507.14397/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2025.

## Built on by (most cited first, 11 of 11)

- [How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf](https://mlsanity.com/paper/2602.18397/) (2026)
- [Computer Architecture's AlphaZero Moment: Automated Discovery in an Encircled World](https://mlsanity.com/paper/2604.03312/) (2026)
- [TriAxialKV: Toward Extreme Low-Precision KV-Cache Quantization for Agentic Inference Tasks](https://mlsanity.com/paper/2605.17170/) (2026)
- [Adaptive and Resource-efficient Agentic AI Systems for Mobile and Embedded Devices: A Survey](https://mlsanity.com/paper/2510.00078/) (2025)
- [TaxBreak: Unmasking the Hidden Costs of LLM Inference Through Overhead Decomposition](https://mlsanity.com/paper/2603.12465/) (2026)
- [TIDE: Temporal Incremental Draft Engine for Self-Improving LLM Inference](https://mlsanity.com/paper/2602.05145/) (2026)
- [Architecture-Aware LLM Inference Optimization on AMD Instinct GPUs: A Comprehensive Benchmark and Deployment Study](https://mlsanity.com/paper/2603.10031/) (2026)
- [Think Before You Grid-Search: Floor-First Triage for LLM Serving](https://mlsanity.com/paper/2607.05876/) (2026)
- [GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems](https://mlsanity.com/paper/2609.14864/) (2026)
- [More Value per Key: Asymmetric Sparse Attention for Faster LLM Decoding](https://mlsanity.com/paper/2610.04753/) (2026)
- [FORGE: Procedural Weight Generation for Sparse MoE Inference](https://mlsanity.com/paper/2610.16890/) (2026)

## Abstract (arXiv)

The rapid advancement of Large Language Models (LLMs) necessitates a deep understanding of their fundamental performance limits. This paper investigates the limits of LLM inference, focusing on hardware-imposed bottlenecks in auto-regressive decoding. We develop LIMINAL, an analytical performance model that abstracts application requirements and hardware capabilities to systematically explore performance and efficiency across a wide range of current, near-future, and hypothetical hardware. We find LIMINAL is accurate when comparing to LLMs executing on existing hardware, achieving a mean absolute error of 7.6%. Our analysis spans from current HBM3 memory technology used in AI accelerators like GPUs and TPUs to systems based on advanced HBM4 and advanced 3D-stacked DRAM technology. We identify five non-negotiable challenges for LLM inference hardware, establishing compute, memory capacity, bandwidth and collective communication as primary barriers to performance. These findings suggest that achieving significant performance gains beyond 10,000 tokens-per-second will require not just hardware evolution but also fundamental algorithmic advances.