mlsanityData to 14 Oct 2026

DANTINOX: A Unified Framework for Multi-Paradigm Language Modeling

arXiv 2609.17535 ↗First posted 11 Jul 2026cs.CLcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Language generation research increasingly spans three paradigms: autoregressive decoding, discrete masked diffusion, and continuous flow-matching. Comparing them is difficult because each lives in a separate codebase, so measured differences often reflect implementation details rather than the paradigms themselves. We present DantinoX, an open-source JAX/Flax library in which a single modular Transformer backbone serves all three paradigms. Switching the generation paradigm, attention mechanism, or hardware topology requires only a configuration change, while the backbone architecture, tokenizer, initialization strategy, and training infrastructure remain consistent. This enables controlled cross-paradigm comparisons within one API for training, streaming inference, and benchmarking.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 39 of its 65 references that are in the map. First posted on arXiv 11 Jul 2026.

Builds on

39 of 65 references.

References · 39

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2016Layer Normalizationcited by 5,669
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  9. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  11. 2020Longformer: The Long-Document Transformercited by 3,413
  12. 2016Pointer Sentinel Mixture Modelscited by 3,096
27 more
  1. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  4. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  6. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  7. 2020GLU Variants Improve Transformercited by 1,804
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  9. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  10. 2019Root Mean Square Layer Normalizationcited by 1,480
  11. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  12. 2018Hierarchical Neural Story Generationcited by 1,193
  13. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  14. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
  15. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  16. 2020On Layer Normalization in the Transformer Architecturecited by 922
  17. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  18. 2019Neural Text Generation with Unlikelihood Trainingcited by 503
  19. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  20. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  21. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 283
  22. 2016Using the Output Embedding to Improve Language Modelscited by 272
  23. 2019A Study of BFLOAT16 for Deep Learning Trainingcited by 232
  24. 2019Show Your Work: Improved Reporting of Experimental Resultscited by 169
  25. 2016Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modelingcited by 92
  26. 2026LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Expertscited by 14
  27. 2025XLM: A Python package for non-autoregressive language modelscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026AsyncLane: Decoupling Refinement from Advancement in Diffusion Language Model Decoding
  2. 2026Posterior Refinement: Fast Language Generation via Any-Order Flow Maps
  3. 2026Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or One
  4. 2026DOS: Dependency-Oriented Sampler for Masked Diffusion Language Models
  5. 2026CD4LM: Consistency Distillation and aDaptive Decoding for Diffusion Language Models
  6. 2024Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding
  7. 2025Speed Always Wins: A Survey on Efficient Architectures for Large Language Models
  8. 2026Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion