# Joint Segmentation and Image Reconstruction with Error Prediction in Photoacoustic Imaging using Deep Learning

arXiv: https://arxiv.org/abs/2407.02653 · HTML: https://mlsanity.com/paper/2407.02653/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 61 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Builds on (most cited first, 24 of 24)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [Bayesian Neural Networks: An Introduction and Survey](https://mlsanity.com/paper/2006.12024/) (2020)
- [How to Start Training: The Effect of Initialization and Architecture](https://mlsanity.com/paper/1803.01719/) (2018)
- [Phase recovery and holographic image reconstruction using deep learning in neural networks](https://mlsanity.com/paper/1705.04286/) (2017)
- [Lensless computational imaging through deep learning](https://mlsanity.com/paper/1702.08516/) (2017)
- [Reliable deep-learning-based phase imaging with uncertainty quantification](https://mlsanity.com/paper/1901.02038/) (2019)
- [Deep Learning in Photoacoustic Tomography: Current approaches and future directions](https://mlsanity.com/paper/2009.07608/) (2020)
- [Low Photon Count Phase Retrieval Using Deep Learning](https://mlsanity.com/paper/1806.10029/) (2018)
- [Deep speckle correlation: a deep learning approach towards scalable imaging through scattering media](https://mlsanity.com/paper/1806.04139/) (2018)
- [Deep learning for biomedical photoacoustic imaging: A review](https://mlsanity.com/paper/2011.02744/) (2020)
- [Deep Learning Microscopy](https://mlsanity.com/paper/1705.04709/) (2017)
- [Moving beyond simulation: data-driven quantitative photoacoustic imaging using tissue-mimicking phantoms](https://mlsanity.com/paper/2306.06748/) (2023)
- [High-Resolution Limited-Angle Phase Tomography of Dense Layered Objects Using Deep Neural Networks](https://mlsanity.com/paper/1812.07380/) (2018)
- [Reconstructing undersampled photoacoustic microscopy images using deep learning](https://mlsanity.com/paper/2006.00251/) (2020)
- [Deep-learning Image Reconstruction for Real-time Photoacoustic System](https://mlsanity.com/paper/2001.04631/) (2020)
- [Improving needle visibility in LED-based photoacoustic imaging using deep learning with semi-synthetic datasets](https://mlsanity.com/paper/2111.07673/) (2021)
- [Deep Learning Enabled Real-Time Photoacoustic Tomography System via Single Data Acquisition Channel](https://mlsanity.com/paper/2001.07454/) (2020)

## Built on by (most cited first, 1 of 1)

- [GauSSmart: Enhanced 3D Reconstruction through 2D Foundation Models and Geometric Filtering](https://mlsanity.com/paper/2510.14270/) (2025)

## Abstract (arXiv)

Deep learning has been used to improve photoacoustic (PA) image reconstruction. One major challenge is that errors cannot be quantified to validate predictions when ground truth is unknown. Validation is key to quantitative applications, especially using limited-bandwidth ultrasonic linear detector arrays. Here, we propose a hybrid Bayesian convolutional neural network (Hybrid-BCNN) to jointly predict PA image and segmentation with error (uncertainty) predictions. Each output pixel represents a probability distribution where error can be quantified. The Hybrid-BCNN was trained with simulated PA data and applied to both simulations and experiments. Due to the sparsity of PA images, segmentation focuses Hybrid-BCNN on minimizing the loss function in regions with PA signals for better predictions. The results show that accurate PA segmentations and images are obtained, and error predictions are highly statistically correlated to actual errors. To leverage error predictions, confidence processing created PA images above a specific confidence level.