# RIS-Kernel: A Model-Agnostic Architecture for Long-Context LLM Inference via Sparse Attention

arXiv: https://arxiv.org/abs/2607.21927 · HTML: https://mlsanity.com/paper/2607.21927/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 31 references that are in the map.
- First posted on arXiv 24 Jul 2026.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [TinyLlama: An Open-Source Small Language Model](https://mlsanity.com/paper/2401.02385/) (2024)
- [YaRN: Efficient Context Window Extension of Large Language Models](https://mlsanity.com/paper/2309.00071/) (2023)
- [The Impact of Positional Encoding on Length Generalization in Transformers](https://mlsanity.com/paper/2305.19466/) (2023)
- [On the Relationship between Self-Attention and Convolutional Layers](https://mlsanity.com/paper/1911.03584/) (2019)
- [Sparse Sinkhorn Attention](https://mlsanity.com/paper/2002.11296/) (2020)
- [Data Movement Is All You Need: A Case Study on Optimizing Transformers](https://mlsanity.com/paper/2007.00072/) (2020)
- [Efficient LLM Inference on CPUs](https://mlsanity.com/paper/2311.00502/) (2023)

## Abstract (arXiv)

Full self-attention in large language models scales as O(N^2), limiting long-context analysis to 65k tokens and requiring costly GPU clusters. The Reduced Interaction Sampling (RIS) inference engine addresses this as a model-agnostic architecture. Without modifying weights, RIS introduces sparse stochastic attention geometries that operate under extreme pruning (up to 99% under 1% density) while preserving factual retrieval. While foundational graph-theoretic RIS formulations establish an asymptotic O(N log N) bound, this work evaluates a practical fixed-density regime on unaccelerated commodity CPU systems. We validate RIS on Qwen2-1.5B-Instruct across two regimes. At 32,768 tokens (where native dense attention serves as the upper bound), RIS-Stochastic at 1% density and 70 ensemble seeds achieves 75.00% accuracy, outperforming the native dense baseline (71.88%), while 5% density at 10 seeds matches it (71.88%), demonstrating that multi-seed sparsity filters sequence-level noise. Crucially, a controlled ablation evaluating a single static mask at matched budget (N=1, d=0.50) yields only 53.12% accuracy, proving that multi-seed topological diversity, rather than raw edge density, drives contextual recovery. Under tight budgets, RIS-Structural reaches 65.62% accuracy at 1% density with 10 seeds (recovering 50.0% of the gap over the 59.38% zero-context floor) and 68.75% (75.0% recovery) with 30 seeds. At 65,536 tokens, where dense attention triggers out-of-memory faults, RIS yields retrieval gains of up to 14.06 percentage points over the zero-context floor (51.56%), confirmed as marginally significant under McNemar's paired test (p = 0.078 < 0.10). All evaluations run on commodity CPU servers (16-128 GB RAM), demonstrating that long-context LLM inference is feasible on standard academic hardware without GPU acceleration.