mlsanityData to 14 Oct 2026

COREY: Entropy-Guided Runtime Chunk Scheduling for Selective Scan Kernels

arXiv 2604.10597 ↗First posted 12 Apr 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Mamba selective state space models (SSMs) provide linear-time sequence modeling but remain sensitive to selective-scan chunk scheduling. We present COREY, a concept-and-feasibility runtime scheduler that maps fixed-bin activation entropy to chunk size. We evaluate COREY in three tiers: a prototype cost model, real-checkpoint kernel timing, and routed end-to-end ablations on modern GPUs. At the kernel level, a calibrated rule, (Href=log K), recovers the locally optimal chunk and matches a one-time static oracle, yielding (4.41×) lower latency than an unoptimized baseline on a consumer GPU and (3.90×)--(4.04×) lower latency on a data-center accelerator. Routing this choice into a patched live scan kernel closes the engineering loop without improving end-to-end speed: in unified routed ablations, the best static chunk outperforms all entropy-guided and proxy schedulers. Sampled-histogram COREY adds (+4.6%) overhead; a guarded fallback to Static-512 reduces this to (+1.3%); and a lightweight sequence-length-keyed table further reduces it to (+0.7%). However, both remain slower than the static oracle because they retain scheduling cost. On an 80-prompt LongBench subset, passive and routed inference are exactly output-equivalent, with (100%) greedy-token agreement and zero metric deltas. A mixed-regime study shows that a single sequence-length rule matches the per-regime chunk oracle for balanced serving. COREY is therefore validated as a quality-preserving scheduling prototype, but current entropy statistics are not a robust throughput win over static chunk tuning on measured SSM checkpoint workloads. SourceCode: https://github.com/mabo1215/COREYTransformer/.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 14 references that are in the map. First posted on arXiv 12 Apr 2026.

Builds on

9 of 14 references.

References · 9

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  3. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  4. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  5. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  6. 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 396
  7. 2022MegaBlocks: Efficient Sparse Training with Mixture-of-Expertscited by 209
  8. 2025MambaQuant: Quantizing the Mamba Family with Variance Aligned Rotation Methodscited by 29
  9. 2024Quamba: A Post-Training Quantization Recipe for Selective State Space Modelscited by 28

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FleetOpt: Analytical Fleet Provisioning for LLM Inference with Compress-and-Route as Implementation Mechanism
  2. 2026Calibrate, Then Route: A Measured Study of Learned Request Routing for Disaggregated LLM Serving
  3. 2026NetKV: Network-Aware Decode Instance Selection for Disaggregated LLM Inference
  4. 2026The 1/W Law: An Analytical Study of Context-Length Routing Topology and GPU Generation Gains for LLM Inference Energy Efficiency
  5. 2026Lightweight Scheduling Policies for Continuously Batched LLM Serving Across GPU Architectures
  6. 2024Saving GPU Hours in LLM Inference System Development and Online Workloads with Simulation and DBMS-Inspired Cache Replacement Policies
  7. 2026MOCAP: Wafer-Scale-Chip-Oriented Memory-Orchestrated Chunked Pipelining Framework for Prefill-Only LLM Inference
  8. 2026Beyond Static RAG: An Adaptive, Tri-Metric Routing Framework for Efficient Long-Context Inference on Commodity GPUs