mlsanityData to 14 Oct 2026

CHIME: A Case for Efficient Long-Context Attention-FC Disaggregated Inference with DIMM-PIM

arXiv 2504.17584 ↗First posted 24 Apr 2025cs.AR cs.LGcited by 5

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Attention-FC Disaggregated (AFD) LLM inference systems offload memory-bound Attention operations to memory-rich accelerators (e.g., CPUs, HBM-PIM) while retaining compute-bound Fully-Connected (FC) operations on GPUs. In this paper, we first design a Disaggregated Roofline Model (DRM) to characterize AFD performance, revealing that system throughput is constrained by the accelerator's limiting factor: either memory bandwidth or capacity. We observe that prior AFD systems often overlook these constraints and fail to balance them, leading to resource underutilization or constrained throughput. Therefore, we propose CHIME, the first AFD system integrating DIMM-PIM, which is a case of the new accelerator that strikes the balance with scalable capacity and bandwidth. To address the synchronization challenges inherent to the distributed cooperating DRAM chips in DIMM-PIM, CHIME employs bubble-free pipelining and hybrid-grained re-layout for efficient attention computation. Furthermore, it maximizes cross-device resource utilization via rankset-granular communication-computation overlapping and alignment-predicting scheduling. Evaluations show CHIME achieves up to 5.15× speedup over state-of-the-art HBM-PIM solutions.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 27 of its 77 references that are in the map. First posted on arXiv 24 Apr 2025.

Builds on

27 of 77 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention23 Mar
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep

Other references · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  4. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  5. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  6. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  7. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
  8. 2024A Survey on Efficient Inference for Large Language Modelscited by 205
  9. 2024Evaluation of OpenAI o1: Opportunities and Challenges of AGIcited by 127
  10. 2020Serving DNNs like Clockwork: Performance Predictability from the Bottom Upcited by 120
  11. 2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
  12. 2024MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUscited by 73
13 more
  1. 2024NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencingcited by 52
  2. 2019RecNMP: Accelerating Personalized Recommendation with Near-Memory Processingcited by 49
  3. 2024FastDecode: High-Throughput GPU-Efficient LLM Serving using Heterogeneous Pipelinescited by 49
  4. 2024NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inferencecited by 37
  5. 2024Duplex: A Device for Large Language Models with Mixture of Experts, Grouped Query Attention, and Continuous Batchingcited by 34
  6. 2025PIM Is All You Need: A CXL-Enabled GPU-Free System for Large Language Model Inferencecited by 22
  7. 2025PAPI: Exploiting Dynamic Parallelism in Large Language Model Decoding with a Processing-In-Memory-Enabled Computing Systemcited by 20
  8. 2024SLO-aware GPU Frequency Scaling for Energy Efficient LLM Inference Servingcited by 19
  9. 2024IANUS: Integrated Accelerator based on NPU-PIM Unified Memory Systemcited by 13
  10. 2021GradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descentcited by 10
  11. 2025Make LLM Inference Affordable to Everyone: Augmenting GPU Memory with NDP-DIMMcited by 6
  12. 2025HPU: High-Bandwidth Processing Unit for Scalable, Cost-effective LLM Inference via GPU Co-processingcited by 5
  13. 2024PIM-MMU: A Memory Management Unit for Accelerating Data Transfers in Commercial PIM Systemscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026TriMoE: Augmenting GPU with AMX-Enabled CPU and DIMM-NDP for High-Throughput MoE Inference via Offloadingcited by 0
  2. 2026TokenStack: A Heterogeneous HBM-PIM Architecture and Runtime for Efficient LLM Inferencecited by 0
  3. 2026Heterogeneous LLM Serving with General-Purpose Processing-Near-Memory for Retrieval-Based Sparse Attentioncited by 0
  4. 2026MoE Expert Execution in Disaggregated LLM Serving with a High-Bandwidth ReRAM Near-Memory Architecturecited by 0
  5. 2026CHIPSMORE: Compute-in-Interconnect and -Memory Chiplets for Multi-Mode Multi-Request LLM Inference Accelerationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving
  2. 2026Rethinking Heterogeneous System Disaggregation for Subquadratic Attention
  3. 2025CHIME: Chiplet-based Heterogeneous Near-Memory Acceleration for Edge Multimodal LLM Inference
  4. 2026VitaLLM: A Versatile and Tiny Accelerator for Mixed-Precision LLM Inference on Edge Devices
  5. 2026Analytical Provisioning for Attention-FFN Disaggregated LLM Serving under Stochastic Workloads
  6. 2026ScoutAttention: Efficient KV Cache Offloading via Layer-Ahead CPU Pre-computation for LLM Inference
  7. 2026HybridGen: Efficient LLM Generative Inference via CPU-GPU Hybrid Computing
  8. 2024MAS-Attention: Memory-Aware Stream Processing for Attention Acceleration on Resource-Constrained Edge Devices