# SEMPAI: a Self-Enhancing Multi-Photon Artificial Intelligence for prior-informed assessment of muscle function and pathology

arXiv: https://arxiv.org/abs/2210.16273 · HTML: https://mlsanity.com/paper/2210.16273/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 88 references that are in the map.
- First posted on arXiv 28 Oct 2022.

## Builds on (most cited first, 16 of 16)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Why gradient clipping accelerates training: A theoretical justification for adaptivity](https://mlsanity.com/paper/1905.11881/) (2019)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning](https://mlsanity.com/paper/2003.04696/) (2020)
- [Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review](https://mlsanity.com/paper/1611.00740/) (2016)
- [A Survey on Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2008.10937/) (2020)
- [BACH: Grand Challenge on Breast Cancer Histology Images](https://mlsanity.com/paper/1808.04277/) (2018)
- [A survey on Deep Learning Advances on Different 3D Data Representations](https://mlsanity.com/paper/1808.01462/) (2018)
- [Learning with Known Operators reduces Maximum Training Error Bounds](https://mlsanity.com/paper/1907.01992/) (2019)
- [Phase Imaging with Computational Specificity (PICS) for measuring dry mass changes in sub-cellular compartments](https://mlsanity.com/paper/2002.08361/) (2020)
- [Illumination Pattern Design with Deep Learning for Single-Shot Fourier Ptychographic Microscopy](https://mlsanity.com/paper/1810.03481/) (2018)
- [Multidimensional Contrast Limited Adaptive Histogram Equalization](https://mlsanity.com/paper/1906.11355/) (2019)
- [DeepTechnome: Mitigating Unknown Bias in Deep Learning Based Assessment of CT Images](https://mlsanity.com/paper/2205.13297/) (2022)

## Abstract (arXiv)

Deep learning (DL) shows notable success in biomedical studies. However, most DL algorithms work as a black box, exclude biomedical experts, and need extensive data. We introduce the Self-Enhancing Multi-Photon Artificial Intelligence (SEMPAI), that integrates hypothesis-driven priors in a data-driven DL approach for research on multiphoton microscopy (MPM) of muscle fibers. SEMPAI utilizes meta-learning to optimize prior integration, data representation, and neural network architecture simultaneously. This allows hypothesis testing and provides interpretable feedback about the origin of biological information in MPM images. SEMPAI performs joint learning of several tasks to enable prediction for small datasets. The method is applied on an extensive multi-study dataset resulting in the largest joint analysis of pathologies and function for single muscle fibers. SEMPAI outperforms state-of-the-art biomarkers in six of seven predictive tasks, including those with scarce data. SEMPAI's DL models with integrated priors are superior to those without priors and to prior-only machine learning approaches.