mlsanityData to 14 Oct 2026

Multimodal Diffusion to Mutually Enhance Polarized Light and Low Resolution EBSD Data

arXiv 2604.22212 ↗First posted 24 Apr 2026eess.IV cs.CV cs.LGcited by 0

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

In spite of the utility of 3-D electron back-scattered diffraction (EBSD) microscopy, the data collection process can be time-consuming with serial-sectioning. Hence, it is natural to look at other modalities, such as polarized light (PL) data, to accelerate EBSD data collection, supplemented with shared information. Complementarily, features in chaotic PL data could even be enriched with a handful of EBSD measurements. To inherently learn the complex dynamics between EBSD and PL to solve these inverse problems, we use an unconditional multimodal diffusion model, motivated by progress in diffusion models for inverse problems. Although trained solely on synthetic data once, our model has strong generalizable capabilities on real data which can be low-resolution, noisy, corrupted, and misregistered. With inference-time scaling, we show gains in performance on a variety of objectives including grain boundary prediction, super-resolution, and denoising. With our model, we demonstrate that there is little difference from full resolution performance with only 25% (1/4 the resolution) of EBSD data and corrupted PL data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 32 of its 55 references that are in the map. First posted on arXiv 24 Apr 2026.

Builds on

32 of 55 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Leveraging Multimodal Diffusion Models to Accelerate Imaging with Side Information7 Oct
  2. 2024Provably Robust Score-Based Diffusion Posterior Sampling for Plug-and-Play Image Reconstruction25 Mar
  3. 2022Diffusion Posterior Sampling for General Noisy Inverse Problems29 Sep

Other references · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Scalable Diffusion Models with Transformerscited by 6,054
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  9. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  10. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  11. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  12. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
17 more
  1. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  2. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  3. 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 344
  4. 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 92
  5. 2025Words or Vision: Do Vision-Language Models Have Blind Faith in Text?cited by 89
  6. 2022Conversion Between CT and MRI Images Using Diffusion and Score-Matching Modelscited by 69
  7. 2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion modelscited by 34
  8. 2025Soft Best-of-n Sampling for Model Alignmentcited by 30
  9. 2025Sample, Don't Search: Rethinking Test-Time Alignment for Language Modelscited by 22
  10. 2022Deep Unfolded Tensor Robust PCA with Self-supervised Learningcited by 7
  11. 2025Generalized Parallel Scaling with Interdependent Generationscited by 6
  12. 2023Optimal Budgeted Rejection Sampling for Generative Modelscited by 5
  13. 2026Jackpot: Optimal Budgeted Rejection Sampling for Extreme Actor-Policy Mismatch Reinforcement Learningcited by 5
  14. 2022How to Adapt Pre-trained Vision-and-Language Models to a Text-only Input?cited by 4
  15. 2024INFusion: Diffusion Regularized Implicit Neural Representations for 2D and 3D accelerated MRI reconstructioncited by 4
  16. 2023A Lightweight Transformer for Faster and Robust EBSD Data Collectioncited by 3
  17. 2026Cross-Modal Guidance for Fast Diffusion-Based Computed Tomographycited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025DIFFRACT: Diffusion-based Restoration via Adaptive Control and Thresholding for Diffraction Imaging
  2. 2023Reference-Free Isotropic 3D EM Reconstruction using Diffusion Models
  3. 2025Improving Multislice Electron Ptychography with a Generative Prior
  4. 2026Parameter Inference and Uncertainty Quantification with Diffusion Models: Extending CDI to 2D Spatial Conditioning
  5. 2025Generative diffusion modeling protocols for improving the Kikuchi pattern indexing in electron back-scatter diffraction
  6. 2024From Diffusion to Resolution: Leveraging 2D Diffusion Models for 3D Super-Resolution Task
  7. 2026Three-Step Conditional Diffusion 3D Reconstruction for Light-Field Microscopy
  8. 2024Denoising diffusion models for high-resolution microscopy image restoration