When Genomic Masking Priors Fail to Transfer: Strong Variant Prediction, Weak Functional Generation
On the map: Diffusion models → Flow matching, time series and science
Abstract (arXiv)
Bidirectional discrete diffusion model appears naturally suited to genomic modeling because it can reconstruct missing sequence from both flanks. We developed GenDA (Genomic Density-optimized Absorbing Diffusion) under the additional hypothesis that entropy-guided span placement would concentrate reconstruction pressure on compositionally complex regions, improving both downstream variant-effect prediction and functional sequence generation. Our results only partially support this premise. After supervised fine-tuning, the 202M-parameter GenDA model reaches a pooled ClinVar SNV AUROC of 0.774, exceeding a similarly scaled autoregressive model by 0.103. However, a matched random-span variant reaches 0.777, providing no evidence that entropy guidance causes the ClinVar improvement. More unexpectedly, GenDA fails a zero-shot functional inpainting stress test: across promoters, enhancers, exon boundaries, and intron boundaries, it does not consistently outperform a control that shuffles the native gap while exactly preserving 3-mer composition. Failure is already present for 50--500-bp gaps, although enhancer degradation worsens at longer gaps. Diagnostics identify several boundary conditions: entropy measures local sequence complexity rather than functional importance; 1-mer tokenization limits physical context; training spans are capped at 300 bp; and high absolute AlphaGenome fidelity can coexist with negative control-normalized restoration. These results show that strong fine-tuned variant prediction, a plausible corruption prior, and functional generation are distinct claims that require separate validation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Flow matching, time series and science, one of 14 directions in Diffusion models. Builds on 11 of its 19 references that are in the map. First posted on arXiv 4 Sep 2026.
Builds on
11 of 19 references.
Closest ancestors · 4
Same direction, newest first
- 2026D3LM: A Discrete DNA Diffusion Language Model for Bidirectional DNA Understanding and Generation2 Mar
- 2025Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Design20 Feb
- 2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design17 Oct
- 2024Dirichlet Flow Matching with Applications to DNA Sequence Design8 Feb
Other references · 7
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
- 2022MaskGIT: Masked Generative Image Transformercited by 1,099
- 2025Large Language Diffusion Modelscited by 912
- 2024Simple and Effective Masked Diffusion Language Modelscited by 848
- 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 194
- 2024Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modelingcited by 113
- 2025Ctrl-DNA: Controllable Cell-Type-Specific Regulatory DNA Design via Constrained RLcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Absorb & Escape: Overcoming Single Model Limitations in Generating Genomic Sequences
- 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Review
- 2024Simple Guidance Mechanisms for Discrete Diffusion Models
- 2024Diffusion on language model encodings for protein sequence generation
- 2025Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertainty
- 2026BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplers
- 2026D2R2: Discrete Diffusion with Regulation Reinforcement for Single-Cell Perturbation Prediction
- 2025GenMol: A Drug Discovery Generalist with Discrete Diffusion