# Robust joint registration of multiple stains and MRI for multimodal 3D histology reconstruction: Application to the Allen human brain atlas

arXiv: https://arxiv.org/abs/2104.14873 · HTML: https://mlsanity.com/paper/2104.14873/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 62 references that are in the map.
- First posted on arXiv 30 Apr 2021.

## Builds on (most cited first, 8 of 8)

- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration](https://mlsanity.com/paper/1805.04605/) (2018)
- [Unsupervised Deformable Registration for Multi-Modal Images via Disentangled Representations](https://mlsanity.com/paper/1903.09331/) (2019)
- [Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration](https://mlsanity.com/paper/2007.02790/) (2020)
- [A model of brain morphological changes related to aging and Alzheimer's disease from cross-sectional assessments](https://mlsanity.com/paper/1905.09826/) (2019)
- [A probabilistic atlas of the human thalamic nuclei combining ex vivo MRI and histology](https://mlsanity.com/paper/1806.08634/) (2018)
- [3D Reconstruction and Segmentation of Dissection Photographs for MRI-free Neuropathology](https://mlsanity.com/paper/2009.05596/) (2020)

## Built on by (most cited first, 4 of 4)

- [Rigid Single-Slice-in-Volume registration via rotation-equivariant 2D/3D feature matching](https://mlsanity.com/paper/2410.18683/) (2024)
- [From Traditional to Deep Learning Approaches in Whole Slide Image Registration: A Methodological Review](https://mlsanity.com/paper/2502.19123/) (2025)
- [A Comprehensive Survey on Magnetic Resonance Image Reconstruction](https://mlsanity.com/paper/2503.07097/) (2025)
- [JUMP: A joint multimodal registration pipeline for neuroimaging with minimal preprocessing](https://mlsanity.com/paper/2401.14250/) (2024)

## Abstract (arXiv)

Joint registration of a stack of 2D histological sections to recover 3D structure (``3D histology reconstruction'') finds application in areas such as atlas building and validation of in vivo imaging. Straightforward pairwise registration of neighbouring sections yields smooth reconstructions but has well-known problems such as ``banana effect'' (straightening of curved structures) and ``z-shift'' (drift). While these problems can be alleviated with an external, linearly aligned reference (e.g., Magnetic Resonance (MR) images), registration is often inaccurate due to contrast differences and the strong nonlinear distortion of the tissue, including artefacts such as folds and tears. In this paper, we present a probabilistic model of spatial deformation that yields reconstructions for multiple histological stains that that are jointly smooth, robust to outliers, and follow the reference shape. The model relies on a spanning tree of latent transforms connecting all the sections and slices of the reference volume, and assumes that the registration between any pair of images can be see as a noisy version of the composition of (possibly inverted) latent transforms connecting the two images. Bayesian inference is used to compute the most likely latent transforms given a set of pairwise registrations between image pairs within and across modalities. The framework is used for accurate 3D reconstruction of two stains (Nissl and parvalbumin) from the Allen human brain atlas, showing its benefits on real data with severe distortions. Moreover, we also provide the registration of the reconstructed volume to MNI space, bridging the gaps between two of the most widely used atlases in histology and MRI. The 3D reconstructed volumes and atlas registration can be downloaded from https://openneuro.org/datasets/ds003590. The code is freely available at https://github.com/acasamitjana/3dhirest.