mlsanityData to 7 Oct 2026

Fast Byte Latent Transformer

arXiv 2605.08044 ↗First posted 8 May 2026cs.CL cs.AI cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Recent byte-level language models (LMs) match the performance of token-level models without relying on subword vocabularies, yet their utility is limited by slow, byte-by-byte autoregressive generation. We address this bottleneck in the Byte Latent Transformer (BLT) through new training and generation techniques. First, we introduce BLT Diffusion (BLT-D), a new model and our fastest BLT variant, trained with an auxiliary block-wise diffusion objective alongside the standard next-byte prediction loss. This enables an inference procedure that generates multiple bytes in parallel per decoding step, substantially reducing the number of forward passes required to generate a sequence. Second, we propose two extensions inspired by speculative decoding that trade some of this speed for higher generation quality: BLT Self-speculation (BLT-S), in which BLT's local decoder continues generating past its normal patch boundaries to draft bytes, which are then verified with a single full-model forward pass; and BLT Diffusion+Verification (BLT-DV), which augments BLT-D with an autoregressive verification step after diffusion-based generation. All methods may achieve an estimated memory-bandwidth cost over 50% lower than BLT on generation tasks. Each approach offers its own unique advantages, together removing key barriers to the practical use of byte-level LMs.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 55 of its 65 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

55 of 65 references.

References · 55

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2021Program Synthesis with Large Language Modelscited by 3,509
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  10. 2020GLU Variants Improve Transformercited by 1,724
  11. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  12. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
38 more
  1. 2019Root Mean Square Layer Normalizationcited by 1,425
  2. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  3. 2025Large Language Diffusion Modelscited by 872
  4. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  5. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  6. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  7. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  8. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  9. 2022Efficiently Scaling Transformer Inferencecited by 529
  10. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  11. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  12. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  13. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  14. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  15. 2023Effective Long-Context Scaling of Foundation Modelscited by 301
  16. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
  17. 2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 229
  18. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 207
  19. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 186
  20. 2023MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformerscited by 164
  21. 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 162
  22. 2021Charformer: Fast Character Transformers via Gradient-based Subword Tokenizationcited by 158
  23. 2023Likelihood-Based Diffusion Language Modelscited by 147
  24. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
  25. 2024Byte Latent Transformer: Patches Scale Better Than Tokenscited by 137
  26. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 128
  27. 2024Tokenization counts: the impact of tokenization on arithmetic in frontier LLMscited by 112
  28. 2021Hierarchical Transformers Are More Efficient Language Modelscited by 99
  29. 2023XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Modelscited by 88
  30. 2020CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characterscited by 87
  31. 2022Efficient Transformers with Dynamic Token Poolingcited by 75
  32. 2020Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERTcited by 69
  33. 2024Getting the most out of your tokenizer for pre-training and domain adaptationcited by 65
  34. 2025Dynamic Chunking for End-to-End Hierarchical Sequence Modelingcited by 65
  35. 2024MambaByte: Token-free Selective State Space Modelcited by 58
  36. 2025Diffusion Language Models are Super Data Learnerscited by 45
  37. 2022What do tokens know about their characters and how do they know it?cited by 36
  38. 2024SpaceByte: Towards Deleting Tokenization from Large Language Modelingcited by 31

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Dynamic Multi-Byte Prediction With Hierarchical Language Modelscited by 0
  2. 2026Easy to anticipate, hard to compute: boundary dependence finds the computed outputs that entropy patching missescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models
  2. 2025Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs
  3. 2024Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion
  4. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decoding
  5. 2025Self Speculative Decoding for Diffusion Large Language Models
  6. 2026Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling
  7. 2026S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation
  8. 2026SimSD: Simple Speculative Decoding in Diffusion Language Models