# Self-Supervised Learning with Generative Adversarial Networks for Electron Microscopy

arXiv: https://arxiv.org/abs/2402.18286 · HTML: https://mlsanity.com/paper/2402.18286/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 28 Feb 2024.

## Builds on (most cited first, 28 of 28)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [Self-Supervised GANs via Auxiliary Rotation Loss](https://mlsanity.com/paper/1811.11212/) (2018)
- [MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray Models](https://mlsanity.com/paper/2010.05352/) (2020)
- [Contrastive Representation Learning for Hand Shape Estimation](https://mlsanity.com/paper/2106.04324/) (2021)
- [Stare at What You See: Masked Image Modeling without Reconstruction](https://mlsanity.com/paper/2211.08887/) (2022)
- [Transfer Learning in Magnetic Resonance Brain Imaging: a Systematic Review](https://mlsanity.com/paper/2102.01530/) (2021)
- [Machine Learning Pipeline for Segmentation and Defect Identification from High Resolution Transmission Electron Microscopy Data](https://mlsanity.com/paper/2001.05022/) (2020)
- [LT-GAN: Self-Supervised GAN with Latent Transformation Detection](https://mlsanity.com/paper/2010.09893/) (2020)
- [Contrastive Spatio-Temporal Pretext Learning for Self-supervised Video Representation](https://mlsanity.com/paper/2112.08913/) (2021)
- [Deep Learning Segmentation of Complex Features in Atomic-Resolution Phase Contrast Transmission Electron Microscopy Images](https://mlsanity.com/paper/2012.05322/) (2020)
- [Understanding the Influence of Receptive Field and Network Complexity in Neural-Network-Guided TEM Image Analysis](https://mlsanity.com/paper/2204.04250/) (2022)
- [Developing and Evaluating Deep Neural Network-based Denoising for Nanoparticle TEM Images with Ultra-low Signal-to-Noise](https://mlsanity.com/paper/2101.07770/) (2021)
- [Segmentation in large-scale cellular electron microscopy with deep learning: A literature survey](https://mlsanity.com/paper/2206.07171/) (2022)
- [Deep Learning of Crystalline Defects from TEM images: A Solution for the Problem of "Never Enough Training Data"](https://mlsanity.com/paper/2307.06322/) (2023)
- [Learning Speaker Embedding with Momentum Contrast](https://mlsanity.com/paper/2001.01986/) (2020)
- [Instance Segmentation of Dislocations in TEM Images](https://mlsanity.com/paper/2309.03499/) (2023)

## Built on by (most cited first, 5 of 5)

- [Noise Calibration and Spatial-Frequency Interactive Network for STEM Image Enhancement](https://mlsanity.com/paper/2504.02555/) (2025)
- [Self-Supervised Learning for Image Segmentation: A Comprehensive Survey](https://mlsanity.com/paper/2505.13584/) (2025)
- [Are Vision Foundation Models Foundational for Electron Microscopy Image Segmentation?](https://mlsanity.com/paper/2602.08505/) (2026)
- [High-Fidelity Synthetic Transmission Electron Microscopy Image Generation Using Diffusion Probabilistic Models for Data-Limited Semiconductor Metrology](https://mlsanity.com/paper/2606.24817/) (2026)
- [Physics-Aligned Self-Supervised Learning for Scientific Imaging](https://mlsanity.com/paper/2607.28868/) (2026)

## Abstract (arXiv)

In this work, we explore the potential of self-supervised learning with Generative Adversarial Networks (GANs) for electron microscopy datasets. We show how self-supervised pretraining facilitates efficient fine-tuning for a spectrum of downstream tasks, including semantic segmentation, denoising, noise & background removal, and super-resolution. Experimentation with varying model complexities and receptive field sizes reveals the remarkable phenomenon that fine-tuned models of lower complexity consistently outperform more complex models with random weight initialization. We demonstrate the versatility of self-supervised pretraining across various downstream tasks in the context of electron microscopy, allowing faster convergence and better performance. We conclude that self-supervised pretraining serves as a powerful catalyst, being especially advantageous when limited annotated data are available and efficient scaling of computational cost is important.