# WhiteMatter: All-to-All Cross-Layer Connections via KV Source Mixing

arXiv: https://arxiv.org/abs/2608.18486 · HTML: https://mlsanity.com/paper/2608.18486/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 42 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://mlsanity.com/paper/2406.17557/) (2024)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [You Only Cache Once: Decoder-Decoder Architectures for Language Models](https://mlsanity.com/paper/2405.05254/) (2024)
- [Reducing Transformer Key-Value Cache Size with Cross-Layer Attention](https://mlsanity.com/paper/2405.12981/) (2024)
- [mHC: Manifold-Constrained Hyper-Connections](https://mlsanity.com/paper/2512.24880/) (2025)
- [Hyper-Connections](https://mlsanity.com/paper/2409.19606/) (2024)
- [Attention Residuals](https://mlsanity.com/paper/2603.15031/) (2026)
- [Layer-Condensed KV Cache for Efficient Inference of Large Language Models](https://mlsanity.com/paper/2405.10637/) (2024)
- [DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averaging](https://mlsanity.com/paper/2402.02622/) (2024)
- [PonderLM: Pretraining Language Models to Ponder in Continuous Space](https://mlsanity.com/paper/2505.20674/) (2025)
- [MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections](https://mlsanity.com/paper/2502.12170/) (2025)
- [Value Residual Learning](https://mlsanity.com/paper/2410.17897/) (2024)
- [MLKV: Multi-Layer Key-Value Heads for Memory Efficient Transformer Decoding](https://mlsanity.com/paper/2406.09297/) (2024)
- [A Systematic Study of Cross-Layer KV Sharing for Efficient LLM Inference](https://mlsanity.com/paper/2410.14442/) (2024)
- [PonderLM-2: Pretraining LLM with Latent Thoughts in Continuous Space](https://mlsanity.com/paper/2509.23184/) (2025)
- [Staircase Attention for Recurrent Processing of Sequences](https://mlsanity.com/paper/2106.04279/) (2021)
- [Utility-Driven Speculative Decoding for Mixture-of-Experts](https://mlsanity.com/paper/2506.20675/) (2025)
- [AdaPonderLM: Gated Pondering Language Models with Token-Wise Adaptive Depth](https://mlsanity.com/paper/2603.01914/) (2026)
- [DeepCrossAttention: Supercharging Transformer Residual Connections](https://mlsanity.com/paper/2502.06785/) (2025)
- [LAuReL: Learned Augmented Residual Layer](https://mlsanity.com/paper/2411.07501/) (2024)
- [Delta Attention Residuals](https://mlsanity.com/paper/2605.18855/) (2026)
- [T^2MLR: Transformer with Temporal Middle-Layer Recurrence](https://mlsanity.com/paper/2607.15178/) (2026)
- [Reconstructing KV Caches with Cross-layer Fusion For Enhanced Transformers](https://mlsanity.com/paper/2512.03870/) (2025)
- [The Recurrent Transformer: Greater Effective Depth and Efficient Decoding](https://mlsanity.com/paper/2604.21215/) (2026)
- [Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior](https://mlsanity.com/paper/2605.26797/) (2026)
- [PonderLM-3: Adaptive Token-Wise Pondering with Differentiable Masking](https://mlsanity.com/paper/2603.02023/) (2026)
- [Stochastic KV Routing: Enabling Adaptive Depth-Wise Cache Sharing](https://mlsanity.com/paper/2604.22782/) (2026)

## Abstract (arXiv)

When generating text, a Transformer produces representations of past tokens at every layer, but each layer can normally use only representations from the same depth. This restriction prevents the model from fully reusing information it has already computed. We introduce WhiteMatter, which allows every layer to draw on past-token representations from any depth. A learned mixer selects the most useful depths for the current context and combines their representations into shared key-value (KV) cache channels. Sharing these channels across layers can reduce the cache size. Given the same number of training tokens, WhiteMatter with a full-size cache performs comparably to a standard Transformer with 50% more layers. With half the KV cache, WhiteMatter outperforms matched standard Transformers at two model scales, up to 1.3B parameters. Cross-layer connections, however, introduce dependencies that slow training and prompt processing. We address this problem with cyclic iteration, which updates interleaved groups of tokens in turn while processing the tokens within each group in parallel. On a reference model trained with exact autoregressive execution, cyclic iteration converges 12.5x faster than standard Jacobi iteration.