mlsanityData to 13 Oct 2026

Express Language Modeling

arXiv 2606.10944 ↗First posted 9 Jun 2026cs.LG cs.DS math.STcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce a new tool, Express, for converting a non-causal attention approximation into a causal approximation with matching approximation guarantees. When combined with the state-of-the-art Thinformer approximation, Express improves upon the best known causal attention guarantees, delivering log3/2(n)/s approximation error with only O(s) memory and O(s2 log2(n)) compression overhead for a sequence of length n. We pair these developments with an efficient I/O-aware Triton implementation, demonstrate substantial speedups over FlashAttention 2, and use Express to overcome four resource bottlenecks in the language modeling pipeline: long-context prefill, KV cache compression, long-form memory-constrained decoding, and long-form compute-constrained decoding.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 30 of its 40 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

30 of 40 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,106
  2. 2024The Llama 3 Herd of Modelscited by 14,486
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,247
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,270
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,716
  6. 2023Let's Verify Step by Stepcited by 3,672
  7. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,637
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,925
  9. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,570
  10. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,137
  11. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,386
  12. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,112
18 more
  1. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 913
  2. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 856
  3. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 515
  4. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 422
  5. 2023HyperAttention: Long-context Attention in Near-Linear Timecited by 85
  6. 2024A Simple and Effective L2 Norm-Based Strategy for KV Cache Compressioncited by 84
  7. 2025Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distributioncited by 72
  8. 2023KDEformer: Accelerating Transformers via Kernel Density Estimationcited by 58
  9. 2025KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environmentscited by 50
  10. 2021Kernel Thinningcited by 47
  11. 2024Softmax is not Enough (for Sharp Size Generalisation)cited by 43
  12. 2021Generalized Kernel Thinningcited by 29
  13. 2020Discrepancy Minimization via a Self-Balancing Walkcited by 22
  14. 2021Distribution Compression in Near-linear Timecited by 19
  15. 2025Streaming Attention Approximation via Discrepancy Theorycited by 9
  16. 2025Low-Rank Thinningcited by 7
  17. 2026WildCat: Near-Linear Attention in Theory and Practicecited by 6
  18. 2026Nearly Optimal Attention Coresetscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025TOGGLE: Temporal Logic-Guided Large Language Model Compression for Edge
  2. 2025Draft-based Approximate Inference for LLMs
  3. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference
  4. 2024DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compaction
  5. 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction
  6. 2026HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inference
  7. 2026A Sparse Glimpse of the Whole: Train-Free Self-Speculative Decoding
  8. 2026End-to-End Context Compression at Scale