# Mitigating Pretraining-Induced Attention Asymmetry in 2D+ Electron Microscopy Image Segmentation

arXiv: https://arxiv.org/abs/2505.14105 · HTML: https://mlsanity.com/paper/2505.14105/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 46 references that are in the map.
- First posted on arXiv 20 May 2025.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [Do Better ImageNet Models Transfer Better?](https://mlsanity.com/paper/1805.08974/) (2018)
- [On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Location](https://mlsanity.com/paper/2003.07064/) (2020)
- [A New 2.5D Representation for Lymph Node Detection using Random Sets of Deep Convolutional Neural Network Observations](https://mlsanity.com/paper/1406.2639/) (2014)
- [Bridging 2D and 3D Segmentation Networks for Computation Efficient Volumetric Medical Image Segmentation: An Empirical Study of 2.5D Solutions](https://mlsanity.com/paper/2010.06163/) (2020)
- [Foundation Models for Biomedical Image Segmentation: A Survey](https://mlsanity.com/paper/2401.07654/) (2024)
- [Which Model to Transfer? Finding the Needle in the Growing Haystack](https://mlsanity.com/paper/2010.06402/) (2020)
- [Mitigating the Bias of Centered Objects in Common Datasets](https://mlsanity.com/paper/2112.09195/) (2021)
- [Cross-dimensional transfer learning in medical image segmentation with deep learning](https://mlsanity.com/paper/2307.15872/) (2023)
- [Black-Box Saliency Map Generation Using Bayesian Optimisation](https://mlsanity.com/paper/2001.11366/) (2020)
- [Consistent Recurrent Neural Networks for 3D Neuron Segmentation](https://mlsanity.com/paper/2102.01021/) (2021)
- [A modular U-Net for automated segmentation of X-ray tomography images in composite materials](https://mlsanity.com/paper/2107.07468/) (2021)
- [Image-Based Correction of Continuous and Discontinuous Non-Planar Axial Distortion in Serial Section Microscopy](https://mlsanity.com/paper/1511.01161/) (2015)

## Abstract (arXiv)

Vision models pretrained on large-scale RGB natural image datasets are widely reused for electron microscopy image segmentation. In electron microscopy, volumetric data are acquired as serial sections and processed as stacks of adjacent grayscale slices, where neighboring slices provide symmetric contextual information for identifying features on the central slice. The common strategy maps such stacks to pseudo-RGB inputs to enable transfer learning from pretrained models. However, this mapping imposes channel-specific semantics inherited from natural images, even though electron microscopy slices are homogeneous in the modality and symmetric in their predictive roles. As a result, pretrained models may encode inductive biases that are misaligned with the inherent symmetry of volumetric electron microscopy data. In this work, it is demonstrated that RGB-pretrained models systematically assign unequal importance to individual input slices when applied to stacked electron microscopy data, despite the absence of any intrinsic channel ordering. Using saliency-based attribution analysis across multiple architectures, a consistent channel-level asymmetry was observed that persists after fine-tuning and affects model interpretability, even when segmentation performance is unchanged. To address this issue, a targeted modification of pretraining weights based on uniform channel initialization was proposed, which restores symmetric feature attribution while preserving the benefits of pretraining. Experiments on the SNEMI, Lucchi and GF-PA66 datasets confirm a substantial reduction in attribution bias without compromising or even improving segmentation accuracy.