mlsanityData to 14 Oct 2026

CHIPSMORE: Compute-in-Interconnect and -Memory Chiplets for Multi-Mode Multi-Request LLM Inference Acceleration

arXiv 2608.30509 ↗First posted 31 Aug 2026cs.ARcited by 0

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Large language model (LLM) inference exhibits substantial variability across adaptation modes, context lengths, and request concurrency, creating challenges for maintaining high utilization, memory efficiency, and scalable performance on compute-in-memory (CIM) accelerators. This paper presents CHIPSMORE, a multi-mode and multi-request LLM inference accelerator that integrates compute-in-interconnect and CIM to support both base-mode and low-rank adaptation (LoRA) inference under diverse workloads. CHIPSMORE employs heterogeneous processing elements consisting of resistive RAM analog compute-in-memory (RRAM-ACIM) and static RAM digital compute-in-memory (SRAM-DCIM) interconnected through a programmable Inter-PE computational network (IPCN). A composable hierarchical key-value (KV) memory scheme dynamically allocates router scratchpad, SRAM-DCIM, and embedded DRAM (eDRAM) resources according to workload requirements, enabling scalable support for long-context and batched inference. Furthermore, a non-replicated multi-request execution pipeline exploits request-level parallelism without duplicating pretrained weights, while a state-aware resource reconfiguration mechanism selectively retains runtime states and power-gates inactive resources to improve energy efficiency. Evaluation using cycle-accurate hardware-software co-simulation demonstrates that CHIPSMORE effectively sustains high throughput across varying model sizes, context lengths, and batch sizes while maintaining favorable power scaling. Compared with Nvidia H100, CHIPSMORE achieves up to 2.38× higher throughput and 27× higher energy efficiency on Mistral-7B inference while eliminating weight replication for multi-request serving.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 14 of its 31 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

14 of 31 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025PIM Is All You Need: A CXL-Enabled GPU-Free System for Large Language Model Inference11 Feb

Other references · 13

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  5. 2022Efficiently Scaling Transformer Inferencecited by 539
  6. 2024NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencingcited by 52
  7. 2023PIM-GPT: A Hybrid Process-in-Memory Accelerator for Autoregressive Transformerscited by 8
  8. 2025CHIME: A Case for Efficient Long-Context Attention-FC Disaggregated Inference with DIMM-PIMcited by 5
  9. 2025LEAP: LLM Inference on Scalable PIM-NoC Architecture with Balanced Dataflow and Fine-Grained Parallelismcited by 4
  10. 2026Ouroboros: Wafer-Scale SRAM CIM with Token-Grained Pipelining for Large Language Model Inferencecited by 3
  11. 2024Efficient transformer adaptation for analog in-memory computing via low-rank adapterscited by 2
  12. 2024ARAS: An Adaptive Low-Cost ReRAM-Based Accelerator for DNNscited by 1
1 more
  1. 2026PRIMAL: Processing-In-Memory Based Low-Rank Adaptation for LLM Inference Acceleratorcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Reimagining Memory Access for LLM Inference: Compression-Aware Memory Controller Design
  2. 2026FlashAccel: Leveraging High-Bandwidth Flash (HBF) for High-Throughput LLM Inference
  3. 2025Architecting Long-Context LLM Acceleration with Packing-Prefetch Scheduler and Ultra-Large Capacity On-Chip Memories
  4. 2023Efficient LLM inference solution on Intel GPU
  5. 2025Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System
  6. 2025SAIL: SRAM-Accelerated LLM Inference System with Lookup-Table-based GEMV
  7. 2026SHIELD: A Segmented Hierarchical Memory Architecture for Energy-Efficient LLM Inference on Edge NPUs
  8. 2025MCBP: A Memory-Compute Efficient LLM Inference Accelerator Leveraging Bit-Slice-enabled Sparsity and Repetitiveness