# Reliable Multimodality Eye Disease Screening via Mixture of Student's t Distributions

arXiv: https://arxiv.org/abs/2303.09790 · HTML: https://mlsanity.com/paper/2303.09790/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 33 references that are in the map.
- First posted on arXiv 17 Mar 2023.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness](https://mlsanity.com/paper/2006.10108/) (2020)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Med3D: Transfer Learning for 3D Medical Image Analysis](https://mlsanity.com/paper/1904.00625/) (2019)
- [Trusted Multi-View Classification](https://mlsanity.com/paper/2102.02051/) (2021)
- [A review: Deep learning for medical image segmentation using multi-modality fusion](https://mlsanity.com/paper/2004.10664/) (2020)
- [Self-supervised Feature Learning via Exploiting Multi-modal Data for Retinal Disease Diagnosis](https://mlsanity.com/paper/2007.11067/) (2020)
- [GAMMA Challenge:Glaucoma grAding from Multi-Modality imAges](https://mlsanity.com/paper/2202.06511/) (2022)
- [Uni4Eye: Unified 2D and 3D Self-supervised Pre-training via Masked Image Modeling Transformer for Ophthalmic Image Classification](https://mlsanity.com/paper/2203.04614/) (2022)
- [Learning Two-Stream CNN for Multi-Modal Age-related Macular Degeneration Categorization](https://mlsanity.com/paper/2012.01879/) (2020)
- [Multimodal Information Fusion for Glaucoma and DR Classification](https://mlsanity.com/paper/2209.00979/) (2022)
- [COROLLA: An Efficient Multi-Modality Fusion Framework with Supervised Contrastive Learning for Glaucoma Grading](https://mlsanity.com/paper/2201.03795/) (2022)
- [Uncertainty Estimation for Multi-view Data: The Power of Seeing the Whole Picture](https://mlsanity.com/paper/2210.02676/) (2022)
- [Calibration of Deep Probabilistic Models with Decoupled Bayesian Neural Networks](https://mlsanity.com/paper/1908.08972/) (2019)

## Built on by (most cited first, 1 of 1)

- [Confidence-aware multi-modality learning for eye disease screening](https://mlsanity.com/paper/2405.18167/) (2024)

## Abstract (arXiv)

Multimodality eye disease screening is crucial in ophthalmology as it integrates information from diverse sources to complement their respective performances. However, the existing methods are weak in assessing the reliability of each unimodality, and directly fusing an unreliable modality may cause screening errors. To address this issue, we introduce a novel multimodality evidential fusion pipeline for eye disease screening, EyeMoSt, which provides a measure of confidence for unimodality and elegantly integrates the multimodality information from a multi-distribution fusion perspective. Specifically, our model estimates both local uncertainty for unimodality and global uncertainty for the fusion modality to produce reliable classification results. More importantly, the proposed mixture of Student's t distributions adaptively integrates different modalities to endow the model with heavy-tailed properties, increasing robustness and reliability. Our experimental findings on both public and in-house datasets show that our model is more reliable than current methods. Additionally, EyeMost has the potential ability to serve as a data quality discriminator, enabling reliable decision-making for multimodality eye disease screening.