# Deep learning facilitates fully automated brain image registration of optoacoustic tomography and magnetic resonance imaging

arXiv: https://arxiv.org/abs/2109.01880 · HTML: https://mlsanity.com/paper/2109.01880/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 54 references that are in the map.
- First posted on arXiv 4 Sep 2021.

## Builds on (most cited first, 7 of 7)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Robust Scene Text Recognition with Automatic Rectification](https://mlsanity.com/paper/1603.03915/) (2016)
- [Improving Deep Learning using Generic Data Augmentation](https://mlsanity.com/paper/1708.06020/) (2017)
- [Classification based Grasp Detection using Spatial Transformer Network](https://mlsanity.com/paper/1803.01356/) (2018)

## Abstract (arXiv)

Multi-spectral optoacoustic tomography (MSOT) is an emerging optical imaging method providing multiplex molecular and functional information from the rodent brain. It can be greatly augmented by magnetic resonance imaging (MRI) that offers excellent soft-tissue contrast and high-resolution brain anatomy. Nevertheless, registration of multi-modal images remains challenging, chiefly due to the entirely different image contrast rendered by these modalities. Previously reported registration algorithms mostly relied on manual user-dependent brain segmentation, which compromised data interpretation and accurate quantification. Here we propose a fully automated registration method for MSOT-MRI multimodal imaging empowered by deep learning. The automated workflow includes neural network-based image segmentation to generate suitable masks, which are subsequently registered using an additional neural network. Performance of the algorithm is showcased with datasets acquired by cross-sectional MSOT and high-field MRI preclinical scanners. The automated registration method is further validated with manual and half-automated registration, demonstrating its robustness and accuracy.