# Towards Automatic Recognition of Pure & Mixed Stones using Intraoperative Endoscopic Digital Images

arXiv: https://arxiv.org/abs/2105.10686 · HTML: https://mlsanity.com/paper/2105.10686/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 33 references that are in the map.
- First posted on arXiv 22 May 2021.

## Builds on (most cited first, 5 of 5)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Assessing deep learning methods for the identification of kidney stones in endoscopic images](https://mlsanity.com/paper/2103.01146/) (2021)
- [EndoL2H: Deep Super-Resolution for Capsule Endoscopy](https://mlsanity.com/paper/2002.05459/) (2020)

## Built on by (most cited first, 15 of 15)

- [On the in vivo recognition of kidney stones using machine learning](https://mlsanity.com/paper/2201.08865/) (2022)
- [Boosting Kidney Stone Identification in Endoscopic Images Using Two-Step Transfer Learning](https://mlsanity.com/paper/2210.13654/) (2022)
- [Improved Kidney Stone Recognition Through Attention and Multi-View Feature Fusion Strategies](https://mlsanity.com/paper/2211.02967/) (2022)
- [On the generalization capabilities of FSL methods through domain adaptation: a case study in endoscopic kidney stone image classification](https://mlsanity.com/paper/2205.00895/) (2022)
- [SegPrompt: Using Segmentation Map as a Better Prompt to Finetune Deep Models for Kidney Stone Classification](https://mlsanity.com/paper/2303.08303/) (2023)
- [Deep Prototypical-Parts Ease Morphological Kidney Stone Identification and are Competitively Robust to Photometric Perturbations](https://mlsanity.com/paper/2304.04077/) (2023)
- [A metric learning approach for endoscopic kidney stone identification](https://mlsanity.com/paper/2307.07046/) (2023)
- [Deep morphological recognition of kidney stones using intra-operative endoscopic digital videos](https://mlsanity.com/paper/2205.06093/) (2022)
- [Comparing feature fusion strategies for Deep Learning-based kidney stone identification](https://mlsanity.com/paper/2206.00069/) (2022)
- [Interpretable Deep Learning Classifier by Detection of Prototypical Parts on Kidney Stones Images](https://mlsanity.com/paper/2206.00252/) (2022)
- [Improving automatic endoscopic stone recognition using a multi-view fusion approach enhanced with two-step transfer learning](https://mlsanity.com/paper/2304.03193/) (2023)
- [Improving Prototypical Parts Abstraction for Case-Based Reasoning Explanations Designed for the Kidney Stone Type Recognition](https://mlsanity.com/paper/2409.12883/) (2024)
- [Evaluating the plausibility of synthetic images for improving automated endoscopic stone recognition](https://mlsanity.com/paper/2409.13409/) (2024)
- [Assessing the generalization performance of SAM for ureteroscopy scene understanding](https://mlsanity.com/paper/2505.17210/) (2025)
- [Evaluation of Few-Shot Learning Methods for Kidney Stone Type Recognition in Ureteroscopy](https://mlsanity.com/paper/2505.17921/) (2025)

## Abstract (arXiv)

Objective: To assess automatic computer-aided in-situ recognition of morphological features of pure and mixed urinary stones using intraoperative digital endoscopic images acquired in a clinical setting. Materials and methods: In this single-centre study, an experienced urologist intraoperatively and prospectively examined the surface and section of all kidney stones encountered. Calcium oxalate monohydrate (COM/Ia), dihydrate (COD/IIb) and uric acid (UA/IIIb) morphological criteria were collected and classified to generate annotated datasets. A deep convolutional neural network (CNN) was trained to predict the composition of both pure and mixed stones. To explain the predictions of the deep neural network model, coarse localisation heat-maps were plotted to pinpoint key areas identified by the network. Results: This study included 347 and 236 observations of stone surface and stone section, respectively. A highest sensitivity of 98 % was obtained for the type "pure IIIb/UA" using surface images. The most frequently encountered morphology was that of the type "pure Ia/COM"; it was correctly predicted in 91 % and 94 % of cases using surface and section images, respectively. Of the mixed type "Ia/COM+IIb/COD", Ia/COM was predicted in 84 % of cases using surface images, IIb/COD in 70 % of cases, and both in 65 % of cases. Concerning mixed Ia/COM+IIIb/UA stones, Ia/COM was predicted in 91 % of cases using section images, IIIb/UA in 69 % of cases, and both in 74 % of cases. Conclusions: This preliminary study demonstrates that deep convolutional neural networks are promising to identify kidney stone composition from endoscopic images acquired intraoperatively. Both pure and mixed stone composition could be discriminated. Collected in a clinical setting, surface and section images analysed by deep CNN provide valuable information about stone morphology for computer-aided diagnosis.