Contrastive Image-Metadata Pre-Training for Materials Transmission Electron Microscopy
Not on the map. Why?
Abstract (arXiv)
The transmission electron microscope facilitates the highest-resolution imaging of any instrument ever created, and its limiting factor is no longer spatial resolution but dose efficiency. Low electron doses avoid sample damage but produce noisy images for which, unlike in classical computer vision, there is no ground truth. Autonomous materials experimentation poses a related problem, since closed-loop instruments need representations grounded in the microscope state at acquisition. Both demand representations grounded in how an image was acquired. We release 7,330 paired high-angle annular dark-field scanning-TEM (HAADF-STEM) images and their seven-dimensional acquisition metadata, and propose Contrastive Image-Metadata Pre-training (CIMP), a CLIP-style encoder that aligns the two modalities and reaches 84.4% Top-1 cross-modal retrieval on a held-out split. All seven parameters are individually recoverable from the frozen visual embedding through a linear probe, and we use the embedding to condition a metadata-conditioned style-transfer model that re-renders experimental images under different acquisition parameters. Virtually scaling dwell time and beam current of low-dose images turns this model into a physics-informed denoiser; in a blind user study, experimental microscopists prefer it over the current state-of-the-art denoiser for STEM imagery on 70.2% of trials.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 45 references that are in the map. First posted on arXiv 27 Apr 2026.
Builds on
15 of 45 references.
References · 15
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
- 2018Demystifying MMD GANscited by 1,293
- 2016Least Squares Generative Adversarial Networkscited by 1,001
- 2018Noise2Noise: Learning Image Restoration without Clean Datacited by 533
- 2018Noise2Void - Learning Denoising from Single Noisy Imagescited by 364
- 2020Unsupervised Deep Video Denoisingcited by 44
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026A Mixture of Experts Foundation Model for Scanning Electron Microscopy Image Analysis
- 2026High-Fidelity Synthetic Transmission Electron Microscopy Image Generation Using Diffusion Probabilistic Models for Data-Limited Semiconductor Metrology
- 2026STEMGym: Benchmarking Sequential Decision-Making under Dose Budgets in Autonomous Electron Microscopy
- 2026Frequency-Aware Dual-Stream Learning for Balanced Realism and Fidelity in Electron Microscopy Imaging
- 2026Improving Combined Detection and Classification of TEM Defects via Mask-Conditioned Latent Diffusion Augmentation
- 2024Preliminary Investigations of a Multi-Faceted Robust and Synergistic Approach in Semiconductor Electron Micrograph Analysis: Integrating Vision Transformers with Large Language and Multimodal Models
- 2023Reference-Free Isotropic 3D EM Reconstruction using Diffusion Models
- 2024Multi-Modal Instruction-Tuning Small-Scale Language-and-Vision Assistant for Semiconductor Electron Micrograph Analysis