mlsanityData to 7 Oct 2026

Triplet-Block Diffusion RWKV

arXiv 2605.25969 ↗First posted 25 May 2026cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Causal Transformer language models suffer from strictly sequential decoding and a quadratic per-step attention cost. While linear-time causal models and discrete diffusion models each address these weaknesses, their integration remains inherently inconsistent: diffusion requires bidirectional attention, while causal models are unidirectional. To unify these architectures, we propose B3D-RWKV, a diffusion RWKV variant that integrates the model's O(L) inference efficiency with parallel, bidirectional discrete-diffusion through a triplet-block layout method. B3D-RWKV-7.2B reaches comparable accuracy on an 8-task suite versus existing models while significantly outperforming baselines in decoding throughput with an average of 1.6× speedup.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 27 of its 32 references that are in the map. First posted on arXiv 25 May 2026.

Builds on

27 of 32 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  8. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  10. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  11. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  12. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
15 more
  1. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  2. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  3. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  4. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  5. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  6. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  7. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  8. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 490
  9. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  10. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 388
  11. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  12. 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 381
  13. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 154
  14. 2025RWKV-7 "Goose" with Expressive Dynamic State Evolutioncited by 134
  15. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inferencecited by 38

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Anatomy of Associative Recall in Fixed-State Recurrences: A Matched-State Decomposition, an Interference Wall, and a Curriculum That Breaks Itcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Causal Autoregressive Diffusion Language Model
  2. 2025Blockwise SFT for Diffusion Language Models: Reconciling Bidirectional Attention and Autoregressive Decoding
  3. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation
  4. 2026Retrofitting Linear Attention into Diffusion Language Models
  5. 2025Encoder-Decoder Diffusion Language Models for Efficient Training and Inference
  6. 2026Rethinking the Generation Order of Block Diffusion Language Models
  7. 2025Beyond Static Cutoffs: One-Shot Dynamic Thresholding for Diffusion Language Models
  8. 2026SimSD: Simple Speculative Decoding in Diffusion Language Models