# Contrast transfer functions help quantify neural network out-of-distribution generalization in HRTEM

arXiv: https://arxiv.org/abs/2512.09067 · HTML: https://mlsanity.com/paper/2512.09067/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 78 references that are in the map.
- First posted on arXiv 9 Dec 2025.

## Builds on (most cited first, 30 of 34)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [PyHessian: Neural Networks Through the Lens of the Hessian](https://mlsanity.com/paper/1912.07145/) (2019)
- [Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization](https://mlsanity.com/paper/2107.04649/) (2021)
- [Domain Generalization using Causal Matching](https://mlsanity.com/paper/2006.07500/) (2020)
- [Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.06607/) (2021)
- [Vision Transformers are Robust Learners](https://mlsanity.com/paper/2105.07581/) (2021)
- [Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization](https://mlsanity.com/paper/2109.02934/) (2021)
- [Diverse Weight Averaging for Out-of-Distribution Generalization](https://mlsanity.com/paper/2205.09739/) (2022)
- [Towards a Theoretical Framework of Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.04496/) (2021)
- [OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.03721/) (2021)
- [Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networks](https://mlsanity.com/paper/2102.12567/) (2021)
- [A statistical framework for efficient out of distribution detection in deep neural networks](https://mlsanity.com/paper/2102.12967/) (2021)
- [A Survey on Evaluation of Out-of-Distribution Generalization](https://mlsanity.com/paper/2403.01874/) (2024)
- [Review: Deep Learning in Electron Microscopy](https://mlsanity.com/paper/2009.08328/) (2020)
- [Visualizing high-dimensional loss landscapes with Hessian directions](https://mlsanity.com/paper/2208.13219/) (2022)
- [Machine Learning Pipeline for Segmentation and Defect Identification from High Resolution Transmission Electron Microscopy Data](https://mlsanity.com/paper/2001.05022/) (2020)
- [LCA-on-the-Line: Benchmarking Out-of-Distribution Generalization with Class Taxonomies](https://mlsanity.com/paper/2407.16067/) (2024)
- [An Empirical Study on Distribution Shift Robustness From the Perspective of Pre-Training and Data Augmentation](https://mlsanity.com/paper/2205.12753/) (2022)
- [Deep Learning Segmentation of Complex Features in Atomic-Resolution Phase Contrast Transmission Electron Microscopy Images](https://mlsanity.com/paper/2012.05322/) (2020)
- [Model-free quantification of completeness, uncertainties, and outliers in atomistic machine learning using information theory](https://mlsanity.com/paper/2404.12367/) (2024)
- [Understanding the Influence of Receptive Field and Network Complexity in Neural-Network-Guided TEM Image Analysis](https://mlsanity.com/paper/2204.04250/) (2022)
- [Developing and Evaluating Deep Neural Network-based Denoising for Nanoparticle TEM Images with Ultra-low Signal-to-Noise](https://mlsanity.com/paper/2101.07770/) (2021)
- [Leveraging generative adversarial networks to create realistic scanning transmission electron microscopy images](https://mlsanity.com/paper/2301.07743/) (2023)
- [Disentangling multiple scattering with deep learning: application to strain mapping from electron diffraction patterns](https://mlsanity.com/paper/2202.00204/) (2022)
- [Ask Your Distribution Shift if Pre-Training is Right for You](https://mlsanity.com/paper/2403.00194/) (2024)

## Abstract (arXiv)

Neural networks, while effective for tackling many challenging scientific tasks, are not known to perform well out-of-distribution (OOD), i.e., within domains which differ from their training data. Understanding neural network OOD generalization is paramount to their successful deployment in experimental workflows, especially when ground-truth knowledge about the experiment is hard to establish or experimental conditions significantly vary. With inherent access to ground-truth information and fine-grained control of underlying distributions, simulation-based data curation facilitates precise investigation of OOD generalization behavior. Here, we probe generalization with respect to imaging conditions of neural network segmentation models for high-resolution transmission electron microscopy (HRTEM) imaging of nanoparticles, training and measuring the OOD generalization of over 12,000 neural networks using synthetic data generated via random structure sampling and multislice simulation. Using the HRTEM contrast transfer function, we further develop a framework to compare information content of HRTEM datasets and quantify OOD domain shifts. We demonstrate that neural network segmentation models enjoy significant performance stability, but will smoothly and predictably worsen as imaging conditions shift from the training distribution. Lastly, we consider limitations of our approach in explaining other OOD shifts, such as of the atomic structures, and discuss complementary techniques for understanding generalization in such settings.