# Let it shine: Autofluorescence of Papanicolaou-stain improves AI-based cytological oral cancer detection

arXiv: https://arxiv.org/abs/2407.01869 · HTML: https://mlsanity.com/paper/2407.01869/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 49 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Builds on (most cited first, 14 of 14)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis](https://mlsanity.com/paper/1804.06353/) (2018)
- [Multiple Instance Learning: A Survey of Problem Characteristics and Applications](https://mlsanity.com/paper/1612.03365/) (2016)
- [MMTM: Multimodal Transfer Module for CNN Fusion](https://mlsanity.com/paper/1911.08670/) (2019)
- [Machine Learning for Integrating Data in Biology and Medicine: Principles, Practice, and Opportunities](https://mlsanity.com/paper/1807.00123/) (2018)
- [Learning with privileged information via adversarial discriminative modality distillation](https://mlsanity.com/paper/1810.08437/) (2018)
- [Fast computation of mutual information in the frequency domain with applications to global multimodal image alignment](https://mlsanity.com/paper/2106.14699/) (2021)
- [A Deep Learning based Pipeline for Efficient Oral Cancer Screening on Whole Slide Images](https://mlsanity.com/paper/1910.10549/) (2019)
- [End-to-end Multiple Instance Learning with Gradient Accumulation](https://mlsanity.com/paper/2203.03981/) (2022)

## Built on by (most cited first, 3 of 3)

- [Isolated Channel Vision Transformers: From Single-Channel Pretraining to Multi-Channel Finetuning](https://mlsanity.com/paper/2503.09826/) (2025)
- [A Comparison of Deep Learning Methods for Cell Detection in Digital Cytology](https://mlsanity.com/paper/2504.06957/) (2025)
- [Preserving Marker Specificity with Lightweight Channel-Independent Representation Learning](https://mlsanity.com/paper/2512.15410/) (2025)

## Abstract (arXiv)

Oral cancer is a global health challenge. It is treatable if detected early, but it is often fatal in late stages. There is a shift from the invasive and time-consuming tissue sampling and histological examination, toward non-invasive brush biopsies and cytological examination. Reliable computer-assisted methods are essential for cost-effective and accurate cytological analysis, but the lack of detailed cell-level annotations impairs model effectiveness. This study aims to improve AI-based oral cancer detection using multimodal imaging and deep fusion. We combine brightfield and fluorescence whole slide microscopy imaging to analyze Papanicolaou-stained liquid-based cytology slides of brush biopsies collected from both healthy and cancer patients. Due to limited cytological annotations, we utilize a weakly supervised deep learning approach using only patient-level labels. We evaluate various multimodal fusion strategies, including early, late, and three recent intermediate fusion methods. Our results show: (i) fluorescence imaging of Papanicolaou-stained samples provides substantial diagnostic information; (ii) multimodal fusion enhances classification and cancer detection accuracy over single-modality methods. Intermediate fusion is the leading method among the studied approaches. Specifically, the Co-Attention Fusion Network (CAFNet) model excels with an F1 score of 83.34% and accuracy of 91.79%, surpassing human performance on the task. Additional tests highlight the need for precise image registration to optimize multimodal analysis benefits. This study advances cytopathology by combining deep learning and multimodal imaging to enhance early, non-invasive detection of oral cancer, improving diagnostic accuracy and streamlining clinical workflows. The developed pipeline is also applicable in other cytological settings. Our codes and dataset are available online for further research.