mlsanityData to 7 Oct 2026

A Mixture of Experts Foundation Model for Scanning Electron Microscopy Image Analysis

arXiv 2604.05960 ↗First posted 7 Apr 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Scanning Electron Microscopy (SEM) is indispensable in modern materials science, enabling high-resolution imaging across a wide range of structural, chemical, and functional investigations. However, SEM imaging remains constrained by task-specific models and labor-intensive acquisition processes that limit its scalability across diverse applications. Here, we introduce the first foundation model for SEM images, pretrained on a large corpus of multi-instrument, multi-condition scientific micrographs, enabling generalization across diverse material systems and imaging conditions. Leveraging a self-supervised transformer architecture, our model learns rich and transferable representations that can be fine-tuned or adapted to a wide range of downstream tasks. As a compelling demonstration, we focus on defocus-to-focus image translation-an essential yet underexplored challenge in automated microscopy pipelines. Our method not only restores focused detail from defocused inputs without paired supervision but also outperforms state-of-the-art techniques across multiple evaluation metrics. This work lays the groundwork for a new class of adaptable SEM models, accelerating materials discovery by bridging foundational representation learning with real-world imaging needs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 54 references that are in the map. First posted on arXiv 7 Apr 2026.

Builds on

15 of 54 references.

References · 15

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  7. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  8. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  9. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  10. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  11. 2020A Survey on Contrastive Self-supervised Learningcited by 553
  12. 2018Noise2Noise: Learning Image Restoration without Clean Datacited by 526
3 more
  1. 2018Noise2Void - Learning Denoising from Single Noisy Imagescited by 359
  2. 2015Deep multi-scale video prediction beyond mean square errorcited by 297
  3. 2023DeepFocus: Fast focus and astigmatism correction for electron microscopycited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025EM3M: An Electron Micrograph Dataset for Microstructural Segmentation and Generation
  2. 2024Foundational Model for Electron Micrograph Analysis: Instruction-Tuning Small-Scale Language-and-Vision Assistant for Enterprise Adoption
  3. 2026High-Fidelity Synthetic Transmission Electron Microscopy Image Generation Using Diffusion Probabilistic Models for Data-Limited Semiconductor Metrology
  4. 2024Multi-Modal Instruction-Tuning Small-Scale Language-and-Vision Assistant for Semiconductor Electron Micrograph Analysis
  5. 2024Parameter-Efficient Quantized Mixture-of-Experts Meets Vision-Language Instruction Tuning for Semiconductor Electron Micrograph Analysis
  6. 2026Improving Combined Detection and Classification of TEM Defects via Mask-Conditioned Latent Diffusion Augmentation
  7. 2024Preliminary Investigations of a Multi-Faceted Robust and Synergistic Approach in Semiconductor Electron Micrograph Analysis: Integrating Vision Transformers with Large Language and Multimodal Models
  8. 2025Novel Concept-Oriented Synthetic Data approach for Training Generative AI-Driven Crystal Grain Analysis Using Diffusion Model