mlsanityData to 14 Oct 2026

PATTON: Enabling Commodity PIM for Production LLM Serving

arXiv 2609.11392 ↗First posted 10 Sep 2026cs.ARcited by 0

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Processing-in-Memory (PIM) is promising for accelerating memory-bound decode attention, but attention acceleration alone is insufficient for production LLM serving, where engines dynamically allocate, populate, share, cache, and reclaim logical KV cache blocks. Supporting this lifecycle on commodity PIM requires efficient physical memory allocation, block-to-address mapping, and command generation. For the Value cache, these requirements create a fundamental conflict among GEMV efficiency, single-token write efficiency, and memory capacity: GEMV-optimized layouts scatter newly generated Value vectors across rows, making writes costly, while finer-grained memory sharing improves capacity utilization but fragments GEMV reductions. We present PATTON, a PIM runtime that integrates production LLM serving engines with commodity PIM. PATTON introduces hierarchical granule allocation: block-sized Key and Value granules map one-to-one to logical token blocks, fixing their physical placements and commands, while coarser granules group blocks for efficient GEMV execution and memory utilization. A Commit Zone stages partial Value blocks for efficient single-token writes before committing them to GEMV-optimized locations. PATTON tracks these placements to generate KV cache writes and QK-transpose/SV commands. Across attention execution and runtime-induced prefill recomputation, PATTON achieves an average 1.95x speedup and 4.83x higher energy efficiency over evaluated baselines, requires no PIM processing-unit modifications, and maintains a KV cache hit rate comparable to the native GPU KV cache in vLLM.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 15 of its 52 references that are in the map. First posted on arXiv 10 Sep 2026.

Builds on

15 of 52 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025PIM Is All You Need: A CXL-Enabled GPU-Free System for Large Language Model Inference11 Feb
  2. 2024PIMphony: Overcoming Bandwidth and Capacity Inefficiency in PIM-based Long-Context LLM Inference System28 Dec
  3. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar
  4. 2023SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills31 Aug

Other references · 11

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  3. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  4. 2024Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttentioncited by 163
  5. 2019TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learningcited by 43
  6. 2021FLAT: An Optimized Dataflow for Mitigating Attention Bottleneckscited by 39
  7. 2023Pathfinding Future PIM Architectures by Demystifying a Commercial PIM Technologycited by 14
  8. 2020Accelerating Bandwidth-Bound Deep Learning Inference with Main-Memory Acceleratorscited by 7
  9. 2022Accelerating Neural Network Inference with Processing-in-DRAM: From the Edge to the Cloudcited by 5
  10. 2022Machine Learning Training on a Real Processing-in-Memory Systemcited by 4
  11. 2025PIM-malloc: A Fast and Scalable Dynamic Memory Allocator for Processing-In-Memory (PIM) Architecturescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TokenStack: A Heterogeneous HBM-PIM Architecture and Runtime for Efficient LLM Inference
  2. 2025Towards Efficient LUT-based PIM: A Scalable and Low-Power Approach for Modern Workloads ↗
  3. 2026AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System
  4. 2024Tackling the Dynamicity in a Production LLM Serving System with SOTA Optimizations via Hybrid Prefill/Decode/Verify Scheduling on Efficient Meta-kernels
  5. 2025Prism: Cost-Efficient Multi-LLM Serving via GPU Memory Ballooning
  6. 2026Blink: CPU-Free LLM Inference by Delegating the Serving Stack to GPU and SmartNIC
  7. 2024APEX: An Extensible and Dynamism-Aware Simulator for Automated Parallel Execution in LLM Serving
  8. 2026Rethinking Unified Memory for NPU-PIM Systems: Dual-View Memory for Dynamic Inference of LLM ↗