# μSplit: efficient image decomposition for microscopy data

arXiv: https://arxiv.org/abs/2211.12872 · HTML: https://mlsanity.com/paper/2211.12872/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 23 Nov 2022.

## Builds on (most cited first, 18 of 18)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Ladder Variational Autoencoders](https://mlsanity.com/paper/1602.02282/) (2016)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images](https://mlsanity.com/paper/2011.10650/) (2020)
- [Cell Detection with Star-convex Polygons](https://mlsanity.com/paper/1806.03535/) (2018)
- ["Double-DIP": Unsupervised Image Decomposition via Coupled Deep-Image-Priors](https://mlsanity.com/paper/1812.00467/) (2018)
- [Taming VAEs](https://mlsanity.com/paper/1810.00597/) (2018)
- [A Poisson-Gaussian Denoising Dataset with Real Fluorescence Microscopy Images](https://mlsanity.com/paper/1812.10366/) (2018)
- [Probabilistic Noise2Void: Unsupervised Content-Aware Denoising](https://mlsanity.com/paper/1906.00651/) (2019)
- [Star-convex Polyhedra for 3D Object Detection and Segmentation in Microscopy](https://mlsanity.com/paper/1908.03636/) (2019)
- [Interpretable Unsupervised Diversity Denoising and Artefact Removal](https://mlsanity.com/paper/2104.01374/) (2021)
- [DenoiSeg: Joint Denoising and Segmentation](https://mlsanity.com/paper/2005.02987/) (2020)
- [Isotropic reconstruction of 3D fluorescence microscopy images using convolutional neural networks](https://mlsanity.com/paper/1704.01510/) (2017)
- [Improving Blind Spot Denoising for Microscopy](https://mlsanity.com/paper/2008.08414/) (2020)
- [Re-parameterizing VAEs for stability](https://mlsanity.com/paper/2106.13739/) (2021)
- [Fully Unsupervised Diversity Denoising with Convolutional Variational Autoencoders](https://mlsanity.com/paper/2006.06072/) (2020)

## Built on by (most cited first, 5 of 5)

- [MicroSSIM: Improved Structural Similarity for Comparing Microscopy Data](https://mlsanity.com/paper/2408.08747/) (2024)
- [denoiSplit: a method for joint microscopy image splitting and unsupervised denoising](https://mlsanity.com/paper/2403.11854/) (2024)
- [HazeMatching: Dehazing Light Microscopy Images with Guided Conditional Flow Matching](https://mlsanity.com/paper/2506.22397/) (2025)
- [scSplit: Bringing Severity Cognizance to Image Decomposition in Fluorescence Microscopy](https://mlsanity.com/paper/2503.22983/) (2025)
- [ResMatching: Noise-Resilient Computational Super-Resolution via Guided Conditional Flow Matching](https://mlsanity.com/paper/2510.26601/) (2025)

## Abstract (arXiv)

We present μSplit, a dedicated approach for trained image decomposition in the context of fluorescence microscopy images. We find that best results using regular deep architectures are achieved when large image patches are used during training, making memory consumption the limiting factor to further improving performance. We therefore introduce lateral contextualization (LC), a novel meta-architecture that enables the memory efficient incorporation of large image-context, which we observe is a key ingredient to solving the image decomposition task at hand. We integrate LC with U-Nets, Hierarchical AEs, and Hierarchical VAEs, for which we formulate a modified ELBO loss. Additionally, LC enables training deeper hierarchical models than otherwise possible and, interestingly, helps to reduce tiling artefacts that are inherently impossible to avoid when using tiled VAE predictions. We apply μSplit to five decomposition tasks, one on a synthetic dataset, four others derived from real microscopy data. Our method consistently achieves best results (average improvements to the best baseline of 2.25 dB PSNR), while simultaneously requiring considerably less GPU memory. Our code and datasets can be found at https://github.com/juglab/uSplit.