Ouroboros: Wafer-Scale SRAM CIM with Token-Grained Pipelining for Large Language Model Inference
Not on the map. Why?
Abstract (arXiv)
Conventional LLM inference architectures suffer from high energy and latency due to frequent data movement across memory hierarchies. We propose Ouroboros, a wafer-scale SRAM-based Computing-in-Memory (CIM) architecture that executes all operations in situ, eliminating off-chip migration. To maximize its limited first-level capacity, we introduce three innovations: Token-Grained Pipelining: Replaces sequence-level pipelining to mitigate length variations, boosting utilization and reducing activation storage. Distributed Dynamic KV Cache Management: Decouples memory from compute to leverage fragmented SRAM for efficient KV storage. Communication-Aware Mapping: Optimizes core allocation for locality and fault tolerance across the wafer. Experimental results show Ouroboros achieves average gains of 4.1× in throughput and 4.2× in energy efficiency, peaking at 9.1× and 17× for the 13B model. (*Due to the notification of arXiv "The Abstract field cannot be longer than 1,920 characters", the appeared Abstract is shortened. For the full Abstract, please download the Article.)
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Mar 2026.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026MOCAP: Wafer-Scale-Chip-Oriented Memory-Orchestrated Chunked Pipelining Framework for Prefill-Only LLM Inferencecited by 0
- 2026MoE Expert Execution in Disaggregated LLM Serving with a High-Bandwidth ReRAM Near-Memory Architecturecited by 0
- 2026CHIPSMORE: Compute-in-Interconnect and -Memory Chiplets for Multi-Mode Multi-Request LLM Inference Accelerationcited by 0
- 2026CHIPSMORE: Compute-in-Interconnect and -Memory Chiplets for Multi-Mode Multi-Request LLM Inference Acceleration31 Aug 2026
- 2026MoE Expert Execution in Disaggregated LLM Serving with a High-Bandwidth ReRAM Near-Memory Architecture14 Aug 2026
- 2026MOCAP: Wafer-Scale-Chip-Oriented Memory-Orchestrated Chunked Pipelining Framework for Prefill-Only LLM Inference22 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026SHIELD: A Segmented Hierarchical Memory Architecture for Energy-Efficient LLM Inference on Edge NPUs
- 2025Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System
- 2025SAIL: SRAM-Accelerated LLM Inference System with Lookup-Table-based GEMV
- 2026Cache-Resident LLM Inference in GB-Scale Last-Level Caches
- 2025Architecting Long-Context LLM Acceleration with Packing-Prefetch Scheduler and Ultra-Large Capacity On-Chip Memories
- 2026TokenStack: A Heterogeneous HBM-PIM Architecture and Runtime for Efficient LLM Inference
- 2025WaferLLM: Large Language Model Inference at Wafer Scale
- 2026KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference