mlsanityData to 7 Oct 2026

FBS: Modeling Native Parallel Reading inside a Transformer

arXiv 2601.21708 ↗First posted 29 Jan 2026cs.AI cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) excel across many tasks, yet inference is still dominated by strictly token-by-token autoregression. Existing acceleration methods largely patch this pipeline and miss core human-reading ingredients: content-adaptive foresight, chunk-structure-aware compute allocation, and train-test consistency for preview/skimming. We propose the Fovea-Block-Skip Transformer (FBS), which injects a causal, trainable loop into Transformers via Parafovea-Attention Window (PAW), Chunk-Head (CH), and Skip-Gate (SG). Across diverse benchmarks, FBS improves the quality-efficiency trade-off without increasing parameters, and ablations show the three modules are complementary.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 43 of its 80 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

43 of 80 references.

References · 43

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2020Longformer: The Long-Document Transformercited by 3,362
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  8. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  9. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  10. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  11. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  12. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
31 more
  1. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  2. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  3. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  4. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 564
  5. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  6. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 434
  7. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 362
  8. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 322
  9. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
  10. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  11. 2023MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformerscited by 164
  12. 2020Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognitioncited by 111
  13. 2023CMATH: Can Your Language Model Pass Chinese Elementary School Math Test?cited by 89
  14. 2017Online and Linear-Time Attention by Enforcing Monotonic Alignmentscited by 85
  15. 2017Monotonic Chunkwise Attentioncited by 82
  16. 2023Lookahead: An Inference Acceleration Framework for Large Language Model with Lossless Generation Accuracycited by 37
  17. 2025Visual Instance-aware Prompt Tuningcited by 29
  18. 2025Think-While-Generating: On-the-Fly Reasoning for Personalized Long-Form Generationcited by 29
  19. 2021FoveaTer: Foveated Transformer for Image Classificationcited by 26
  20. 2025Adaptive Layer-skipping in Pre-trained LLMscited by 24
  21. 2024FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skippingcited by 23
  22. 2022Perplexed by Quality: A Perplexity-based Method for Adult and Harmful Content Detection in Multilingual Heterogeneous Web Datacited by 22
  23. 2025Set Block Decoding is a Language Model Inference Acceleratorcited by 22
  24. 2025Prompt-based Adaptation in Large-scale Vision Models: A Surveycited by 21
  25. 2019PKUSEG: A Toolkit for Multi-Domain Chinese Word Segmentationcited by 17
  26. 2026Double: Breaking the Acceleration Limit via Double Retrieval Speculative Parallelismcited by 15
  27. 2025PDTrim: Targeted Pruning for Prefill-Decode Disaggregation in Inferencecited by 14
  28. 2025TrimTokenator-LC: Towards Adaptive Visual Token Pruning for Large Multimodal Models with Long Contextscited by 14
  29. 2025SlimInfer: Accelerating Long-Context LLM Inference via Dynamic Token Pruningcited by 13
  30. 2023Chunk, Align, Select: A Simple Long-sequence Processing Method for Transformerscited by 9
  31. 2023YUAN 2.0: A Large Language Model with Localized Filtering-based Attentioncited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Air-Know: Arbiter-Calibrated Knowledge-Internalizing Robust Network for Composed Image Retrievalcited by 17
  2. 2026ConeSep: Cone-based Robust Noise-Unlearning Compositional Network for Composed Image Retrievalcited by 15
  3. 2026TEMA: Anchor the Image, Follow the Text for Multi-Modification Composed Image Retrievalcited by 13
  4. 2026MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolutioncited by 12
  5. 2026FoE: Forest of Errors Makes the First Solution the Best in Large Reasoning Modelscited by 8
  6. 2026Sparse Growing Transformer: Training-Time Sparse Depth Allocation via Progressive Attention Loopingcited by 3