mlsanityData to 7 Oct 2026

CORE: Context-Robust Remasking for Diffusion Language Models

arXiv 2602.04096 ↗First posted 4 Feb 2026cs.LGcited by 10

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Standard decoding in Masked Diffusion Models (MDMs) is hindered by context rigidity: tokens are retained based on transient high confidence, often ignoring that early predictions lack full context. This creates cascade effects where initial inconsistencies misguide the remaining generation. Existing revision strategies attempt to mitigate this by relying on static confidence scores, but these signals are inherently myopic; inconsistent tokens can appear confident to the model itself. We propose Context-Robust Remasking (CORE), a training-free framework for inference-time revision. Rather than trusting static token probabilities, CORE identifies context-brittle tokens by probing their sensitivity to targeted masked-context perturbations. We formalize revision as a robust optimization objective over context shifts and efficiently approximate this objective to prioritize unstable tokens for revision. On LLaDA-8B-Base, CORE delivers consistent improvements across reasoning and code benchmarks, outperforming compute-matched baselines and improving MBPP by up to 9.2 percentage points.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 37 of its 44 references that are in the map. First posted on arXiv 4 Feb 2026.

Builds on

37 of 44 references.

Closest ancestors · 17

Same direction, newest first

  1. 2026Deferred Commitment Decoding for Diffusion Language Models5 Jan
  2. 2025ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding15 Dec
  3. 2025Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models26 Nov
  4. 2025Variational Masked Diffusion Models27 Oct
  5. 2025Fine-Tuning Masked Diffusion for Provable Self-Correction1 Oct
  6. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models28 Sep
  7. 2025Any-Order Flexible Length Masked Diffusion31 Aug
  8. 2025Dream 7B: Diffusion Large Language Models21 Aug
  9. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  10. 2025Generalized Interpolating Discrete Diffusion6 Mar
  11. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  12. 2025Large Language Diffusion Models14 Feb
5 more
  1. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  2. 2025Path Planning for Masked Diffusion Model Sampling5 Feb
  3. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  4. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  5. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 20

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  8. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  10. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  11. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  12. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
8 more
  1. 2017Non-Autoregressive Neural Machine Translationcited by 457
  2. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Modelscited by 201
  3. 2019Levenshtein Transformercited by 197
  4. 2019Insertion Transformer: Flexible Sequence Generation via Insertion Operationscited by 123
  5. 2024Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Predictioncited by 28
  6. 2025Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Designcited by 21
  7. 2025Test-Time Scaling in Diffusion LLMs via Hidden Semi-Autoregressive Expertscited by 8
  8. 2025Review, Remask, Refine (R3): Process-Guided Block Diffusion for Text Generationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026AnchorDiff: Topology-Aware Masked Diffusion with Confidence-based Rewriting for Radiology Report Generationcited by 2
  2. 2026Don't Commit Alone: Joint Token Commitment in Diffusion Language Modelscited by 2
  3. 2026Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Language Modelscited by 1
  4. 2026CaRE Compute-aware Remasking Evaluation Protocol for Masked Diffusion Language Modelscited by 1
  5. 2026D3Seg: Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalitiescited by 0
  6. 2026Looped Diffusion Language Modelscited by 0
  7. 2026Targeted Remasking: Replacing Token Editing with Token-to-Mask Refinement in Discrete Diffusion Language Modelscited by 0
  8. 2026Unified Energy for Invariant and Independent Decoding in Diffusion Language Modelscited by 0
  9. 2026Commit Locally, Exit Globally: Coordinating Adaptive Sampling and Early Exit in Diffusion Language Modelscited by 0
  10. 2026A Ticket from Marginals to Joints: Coupled-Noise Distillation for One-Step Block Generation in Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CoDR: Training-Free Confidence-Drift Remasking for Diffusion Language Models
  2. 2026Re-evaluating Confidence Remasking in Masked Diffusion Language Models
  3. 2026NAVIRA: Decoupled Stochastic Remasking for Masked Diffusion Language Models
  4. 2025STaRR: Spatial-Temporal Token-Dynamics-Aware Responsive Remasking for Diffusion Language Models
  5. 2026Revise, Don't Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Models
  6. 2026Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decoding
  7. 2026Know When to Hold 'em: Correct-Token Retention in Uniform-State Diffusion Language Models
  8. 2026Learn from Your Mistakes: Self-Correcting Masked Diffusion Models