mlsanityData to 7 Oct 2026

Post-Boundary Bridge: Must Local Attention Go Global Between Global Layers?

arXiv 2606.02680 ↗First posted 1 Jun 2026cs.LGcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Hybrid Transformers reduce the cost of long-context modeling by combining local attention with periodic full-attention layers. When global communication is already available, however, the best use of local computation remains unclear. We introduce Post-Boundary Bridge (PBB), which preserves causal attention within blocks and adds direct connections across their boundaries. Rather than extending the range of information relayed through successive local layers, PBB prioritizes within-block modeling and nearby exchange, leaving long-range communication to full-attention layers. Across dense and mixture-of-experts models with 205 million to 2.07 billion stored parameters, PBB hybrids retain near-Full perplexity and competitive performance on standard downstream benchmarks while improving controlled source retrieval. At 205 million parameters, PBB also matches a hybrid using sliding-window attention (SWA) in perplexity and achieves higher source-retrieval accuracy. The same boundary-aligned structure enables Flash-PBB, an exact decoding implementation that updates its key-value cache without relocating retained entries. Compared with Flash-SWA, Flash-PBB delivers 1.82x decode attention-core throughput with half the allocated local key-value cache. These results show that targeted boundary exchange can preserve model quality while enabling faster, more memory-efficient local attention between global layers.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 36 of its 47 references that are in the map. First posted on arXiv 1 Jun 2026.

Builds on

36 of 47 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads14 Oct
  2. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun
  3. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs3 Oct

Other references · 33

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  5. 2020Longformer: The Long-Document Transformercited by 3,362
  6. 2023Mistral 7Bcited by 3,065
  7. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  8. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  9. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  10. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  11. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  12. 2020Reformer: The Efficient Transformercited by 1,658
21 more
  1. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  2. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  3. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  4. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  5. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  6. 2018Image Transformercited by 770
  7. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
  8. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 553
  9. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 495
  10. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  11. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 391
  12. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 375
  13. 2023LongLoRA: Efficient Fine-tuning of Long-Context Large Language Modelscited by 243
  14. 2023Landmark Attention: Random-Access Infinite Context Length for Transformerscited by 206
  15. 2023LongNet: Scaling Transformers to 1,000,000,000 Tokenscited by 201
  16. 2024Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attentioncited by 194
  17. 2025MoBA: Mixture of Block Attention for Long-Context LLMscited by 188
  18. 2023MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformerscited by 164
  19. 2025NoLiMa: Long-Context Evaluation Beyond Literal Matchingcited by 120
  20. 2022Block-Recurrent Transformerscited by 107
  21. 2025PowerAttention: Exponentially Scaling of Receptive Fields for Effective Sparse Attentioncited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference
  2. 2026FlashBlock: Attention Caching for Efficient Long-Context Block Diffusion
  3. 2026HySparse2: Hybrid Sparse Attention with Two-Level KV Sharing
  4. 2024Block Transformer: Global-to-Local Language Modeling for Fast Inference
  5. 2026Double-P: Hierarchical Top-P Sparse Attention for Long-Context LLMs
  6. 2026Rethinking the Role of Efficient Attention in Hybrid Architectures
  7. 2026AB-Sparse: Sparse Attention with Adaptive Block Size for Accurate and Efficient Long-Context Inference
  8. 2026SinkRouter: Sink-Aware Routing for Efficient Long-Context Decoding in Large Language and Multimodal Models