mlsanityData to 7 Oct 2026

Diffutron: A Masked Diffusion Language Model for Turkish Language

arXiv 2603.20466 ↗First posted 20 Mar 2026cs.CL cs.AIcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Masked Diffusion Language Models (MDLMs) have emerged as a compelling non-autoregressive alternative to standard large language models; however, their application to morphologically rich languages remains limited. In this paper, we introduce textitDiffutron, a masked diffusion language model specifically designed for Turkish. Our approach leverages a resource-efficient training pipeline, starting with LoRA-based continual pre-training of a multilingual encoder on a large-scale corpus. To enable generative capabilities, we employ a progressive instruction-tuning strategy, sequentially adapting the model on general and task-specific instruction sets. Experimental results across comprehensive benchmarks demonstrate that, despite its compact size, our model achieves competitive performance compared to existing multi-billion-parameter baselines. These findings validate the effectiveness of masked diffusion modeling combined with multi-stage tuning for non-autoregressive text generation in Turkish.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 12 of its 24 references that are in the map. First posted on arXiv 20 Mar 2026.

Builds on

12 of 24 references.

References · 12

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  5. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  6. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  7. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  8. 2020A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languagescited by 131
  9. 2024Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuningcited by 59
  10. 2024TURNA: A Turkish Encoder-Decoder Language Model for Enhanced Understanding and Generationcited by 13
  11. 2025Rethinking Multilingual Continual Pretraining: Data Mixing for Adapting LLMs Across Languages and Resourcescited by 10
  12. 2025Cetvel: A Unified Benchmark for Evaluating Language Understanding, Generation and Cultural Capacity of LLMs for Turkishcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Simple and Effective Masked Diffusion Language Models
  2. 2024Scaling up Masked Diffusion Models on Text
  3. 2023A Cheaper and Better Diffusion Language Model with Soft-Masked Noise
  4. 2026CRoCoDiL: Continuous and Robust Conditioned Diffusion for Language
  5. 2026Representation-based Masked Diffusion Model
  6. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models
  7. 2026Understanding and Accelerating the Training of Masked Diffusion Language Models
  8. 2026DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling