The unreasonable effectiveness of AI CADe polyp detectors to generalize to new countries
Not on the map. Why?
Abstract (arXiv)
Background and aims: Artificial Intelligence (AI) Computer-Aided Detection (CADe) is commonly used for polyp detection, but data seen in clinical settings can differ from model training. Few studies evaluate how well CADe detectors perform on colonoscopies from countries not seen during training, and none are able to evaluate performance without collecting expensive and time-intensive labels. Methods: We trained a CADe polyp detector on Israeli colonoscopy videos (5004 videos, 1106 hours) and evaluated on Japanese videos (354 videos, 128 hours) by measuring the True Positive Rate (TPR) versus false alarms per minute (FAPM). We introduce a colonoscopy dissimilarity measure called "MAsked mediCal Embedding Distance" (MACE) to quantify differences between colonoscopies, without labels. We evaluated CADe on all Japan videos and on those with the highest MACE. Results: MACE correctly quantifies that narrow-band imaging (NBI) and chromoendoscopy (CE) frames are less similar to Israel data than Japan whitelight (bootstrapped z-test, |z| > 690, p < 10-8 for both). Despite differences in the data, CADe performance on Japan colonoscopies was non-inferior to Israel ones without additional training (TPR at 0.5 FAPM: 0.957 and 0.972 for Israel and Japan; TPR at 1.0 FAPM: 0.972 and 0.989 for Israel and Japan; superiority test t > 45.2, p < 10-8). Despite not being trained on NBI or CE, TPR on those subsets were non-inferior to Japan overall (non-inferiority test t > 47.3, p < 10-8, δ = 1.5% for both). Conclusion: Differences that prevent CADe detectors from performing well in non-medical settings do not degrade the performance of our AI CADe polyp detector when applied to data from a new country. MACE can help medical AI models internationalize by identifying the most "dissimilar" data on which to evaluate models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 46 references that are in the map. First posted on arXiv 11 Dec 2023.
Builds on
16 of 46 references.
References · 16
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2016Equality of Opportunity in Supervised Learningcited by 2,007
- 2019Similarity of Neural Network Representations Revisitedcited by 1,755
- 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
- 2018Domain Adaptive Faster R-CNN for Object Detection in the Wildcited by 443
- 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
- 2017Unified Deep Supervised Domain Adaptation and Generalizationcited by 323
- 2022Masked Siamese Networks for Label-Efficient Learningcited by 302
- 2018Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptationcited by 218
- 2020Contrastive Training for Improved Out-of-Distribution Detectioncited by 186
- 2020Towards Learning a Universal Non-Semantic Representation of Speechcited by 84
4 more
- 2019Personalizing ASR for Dysarthric and Accented Speech with Limited Datacited by 45
- 2021Universal Paralinguistic Speech Representations Using Self-Supervised Conformerscited by 32
- 2022The Need for Medically Aware Video Compression in Gastroenterologycited by 3
- 2023Semantic Parsing of Colonoscopy Videos with Multi-Label Temporal Networkscited by 3
Built on by
Not cited yet.