# PyTorch Connectomics: A Scalable and Flexible Segmentation Framework for EM Connectomics

arXiv: https://arxiv.org/abs/2112.05754 · HTML: https://mlsanity.com/paper/2112.05754/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 57 references that are in the map.
- First posted on arXiv 10 Dec 2021.

## Builds on (most cited first, 19 of 19)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [Rethinking Pre-training and Self-training](https://mlsanity.com/paper/2006.06882/) (2020)
- [Superhuman Accuracy on the SNEMI3D Connectomics Challenge](https://mlsanity.com/paper/1706.00120/) (2017)
- [A Multigrid Method for Efficiently Training Video Models](https://mlsanity.com/paper/1912.00998/) (2019)
- [Star-convex Polyhedra for 3D Object Detection and Segmentation in Microscopy](https://mlsanity.com/paper/1908.03636/) (2019)
- [NucMM Dataset: 3D Neuronal Nuclei Instance Segmentation at Sub-Cubic Millimeter Scale](https://mlsanity.com/paper/2107.05840/) (2021)
- [Synaptic Cleft Segmentation in Non-Isotropic Volume Electron Microscopy of the Complete Drosophila Brain](https://mlsanity.com/paper/1805.02718/) (2018)
- [AxonEM Dataset: 3D Axon Instance Segmentation of Brain Cortical Regions](https://mlsanity.com/paper/2107.05451/) (2021)
- [Advanced Deep Networks for 3D Mitochondria Instance Segmentation](https://mlsanity.com/paper/2104.07961/) (2021)
- [CleftNet: Augmented Deep Learning for Synaptic Cleft Detection from Brain Electron Microscopy](https://mlsanity.com/paper/2101.04266/) (2021)
- [A Topological Nomenclature for 3D Shape Analysis in Connectomics](https://mlsanity.com/paper/1909.12887/) (2019)

## Built on by (most cited first, 3 of 3)

- [Instance Segmentation of Unlabeled Modalities via Cyclic Segmentation GAN](https://mlsanity.com/paper/2204.03082/) (2022)
- [Cyto R-CNN and CytoNuke Dataset: Towards reliable whole-cell segmentation in bright-field histological images](https://mlsanity.com/paper/2401.15638/) (2024)
- [FGNet: Leveraging Feature-Guided Attention to Refine SAM2 for 3D EM Neuron Segmentation](https://mlsanity.com/paper/2511.13063/) (2025)

## Abstract (arXiv)

We present PyTorch Connectomics (PyTC), an open-source deep-learning framework for the semantic and instance segmentation of volumetric microscopy images, built upon PyTorch. We demonstrate the effectiveness of PyTC in the field of connectomics, which aims to segment and reconstruct neurons, synapses, and other organelles like mitochondria at nanometer resolution for understanding neuronal communication, metabolism, and development in animal brains. PyTC is a scalable and flexible toolbox that tackles datasets at different scales and supports multi-task and semi-supervised learning to better exploit expensive expert annotations and the vast amount of unlabeled data during training. Those functionalities can be easily realized in PyTC by changing the configuration options without coding and adapted to other 2D and 3D segmentation tasks for different tissues and imaging modalities. Quantitatively, our framework achieves the best performance in the CREMI challenge for synaptic cleft segmentation (outperforms existing best result by relatively 6.1%) and competitive performance on mitochondria and neuronal nuclei segmentation. Code and tutorials are publicly available at https://connectomics.readthedocs.io.