mlsanityData to 14 Oct 2026

UniCAIM: A Unified CAM/CIM Architecture with Static-Dynamic KV Cache Pruning for Efficient Long-Context LLM Inference

arXiv 2504.07479 ↗First posted 10 Apr 2025cs.ARcited by 2

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Transformer-based large language models (LLMs) have achieved impressive performance in various natural language processing (NLP) applications. However, the high memory and computation cost induced by the KV cache limits the inference efficiency, especially for long input sequences. Compute-in-memory (CIM)-based accelerators have been proposed for LLM acceleration with KV cache pruning. However, as existing accelerators only support static pruning with a fixed pattern or dynamic pruning with primitive implementations, they suffer from either high accuracy degradation or low efficiency. In this paper, we propose a ferroelectric FET (FeFET)-based unified content addressable memory (CAM) and CIM architecture, dubbed as UniCAIM. UniCAIM features simultaneous support for static and dynamic pruning with 3 computation modes: 1) in the CAM mode, UniCAIM enables approximate similarity measurement in O(1) time for dynamic KV cache pruning with high energy efficiency; 2) in the charge-domain CIM mode, static pruning can be supported based on accumulative similarity score, which is much more flexible compared to fixed patterns; 3) in the current-domain mode, exact attention computation can be conducted with a subset of selected KV cache. We further propose a novel CAM/CIM cell design that leverages the multi-level characteristics of FeFETs for signed multibit storage of the KV cache and in-place attention computation. With extensive experimental results, we demonstrate UniCAIM can reduce the area-energy-delay product (AEDP) by 8.2-831x over the state-ofthe-art CIM-based LLM accelerators at the circuit level, along with high accuracy comparable with dense attention at the application level, showing its great potential for efficient long-context LLM inference.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 16 of its 39 references that are in the map. First posted on arXiv 10 Apr 2025.

Builds on

16 of 39 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption25 Jul
  2. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun
  3. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  4. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 12

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Longformer: The Long-Document Transformercited by 3,413
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  4. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  5. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 383
  6. 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 246
  7. 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 234
  8. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
  9. 2023MarioGPT: Open-Ended Text2Level Generation through Large Language Modelscited by 64
  10. 2024ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Cachingcited by 48
  11. 2018Efficient and Robust Parallel DNN Training through Model Parallelism on Multi-GPU Platformcited by 28
  12. 2022Sparse Attention Acceleration with Synergistic In-Memory Pruning and On-Chip Recomputationcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026HiKV: Hierarchical Importance-Aware KV Cache with Hardware Acceleration for LLM Decodingcited by 0
  2. 2026SPLASH: Co-Designing Sparse Attention with High-Bandwidth Flash for Efficient Long-Context Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Titanus: Enabling KV Cache Pruning and Quantization On-the-Fly for LLM Acceleration
  2. 2025MCBP: A Memory-Compute Efficient LLM Inference Accelerator Leveraging Bit-Slice-enabled Sparsity and Repetitiveness
  3. 2025VEDA: Efficient LLM Generation Through Voting-based KV Cache Eviction and Dataflow-flexible Accelerator
  4. 2024A Survey on Large Language Model Acceleration based on KV Cache Management
  5. 2026Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy
  6. 2025Accelerating LLM Inference with Flexible N:M Sparsity via A Fully Digital Compute-in-Memory Accelerator
  7. 2026Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
  8. 2025Mustafar: Promoting Unstructured Sparsity for KV Cache Pruning in LLM Inference