# On the in vivo recognition of kidney stones using machine learning

arXiv: https://arxiv.org/abs/2201.08865 · HTML: https://mlsanity.com/paper/2201.08865/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 74 references that are in the map.
- First posted on arXiv 21 Jan 2022.

## Builds on (most cited first, 11 of 11)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020)
- [Domain Adaptation for Medical Image Analysis: A Survey](https://mlsanity.com/paper/2102.09508/) (2021)
- [Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning](https://mlsanity.com/paper/1609.06570/) (2016)
- [An Analysis of Deep Neural Network Models for Practical Applications](https://mlsanity.com/paper/1605.07678/) (2016)
- [End-to-End Trainable Deep Active Contour Models for Automated Image Segmentation: Delineating Buildings in Aerial Imagery](https://mlsanity.com/paper/2007.11691/) (2020)
- [Towards Automatic Recognition of Pure & Mixed Stones using Intraoperative Endoscopic Digital Images](https://mlsanity.com/paper/2105.10686/) (2021)

## Built on by (most cited first, 14 of 14)

- [Boosting Kidney Stone Identification in Endoscopic Images Using Two-Step Transfer Learning](https://mlsanity.com/paper/2210.13654/) (2022)
- [Improved Kidney Stone Recognition Through Attention and Multi-View Feature Fusion Strategies](https://mlsanity.com/paper/2211.02967/) (2022)
- [SegPrompt: Using Segmentation Map as a Better Prompt to Finetune Deep Models for Kidney Stone Classification](https://mlsanity.com/paper/2303.08303/) (2023)
- [Deep Prototypical-Parts Ease Morphological Kidney Stone Identification and are Competitively Robust to Photometric Perturbations](https://mlsanity.com/paper/2304.04077/) (2023)
- [A metric learning approach for endoscopic kidney stone identification](https://mlsanity.com/paper/2307.07046/) (2023)
- [Comparing feature fusion strategies for Deep Learning-based kidney stone identification](https://mlsanity.com/paper/2206.00069/) (2022)
- [Interpretable Deep Learning Classifier by Detection of Prototypical Parts on Kidney Stones Images](https://mlsanity.com/paper/2206.00252/) (2022)
- [Improving automatic endoscopic stone recognition using a multi-view fusion approach enhanced with two-step transfer learning](https://mlsanity.com/paper/2304.03193/) (2023)
- [Improving Prototypical Parts Abstraction for Case-Based Reasoning Explanations Designed for the Kidney Stone Type Recognition](https://mlsanity.com/paper/2409.12883/) (2024)
- [Causal Scoring Medical Image Explanations: A Case Study On Ex-vivo Kidney Stone Images](https://mlsanity.com/paper/2309.01921/) (2023)
- [Evaluating the plausibility of synthetic images for improving automated endoscopic stone recognition](https://mlsanity.com/paper/2409.13409/) (2024)
- [Assessing the generalization performance of SAM for ureteroscopy scene understanding](https://mlsanity.com/paper/2505.17210/) (2025)
- [Evaluation of Few-Shot Learning Methods for Kidney Stone Type Recognition in Ureteroscopy](https://mlsanity.com/paper/2505.17921/) (2025)
- [Vision Transformers for Kidney Stone Image Classification: A Comparative Study with CNNs](https://mlsanity.com/paper/2508.13461/) (2025)

## Abstract (arXiv)

Determining the type of kidney stones allows urologists to prescribe a treatment to avoid recurrence of renal lithiasis. An automated in-vivo image-based classification method would be an important step towards an immediate identification of the kidney stone type required as a first phase of the diagnosis. In the literature it was shown on ex-vivo data (i.e., in very controlled scene and image acquisition conditions) that an automated kidney stone classification is indeed feasible. This pilot study compares the kidney stone recognition performances of six shallow machine learning methods and three deep-learning architectures which were tested with in-vivo images of the four most frequent urinary calculi types acquired with an endoscope during standard ureteroscopies. This contribution details the database construction and the design of the tested kidney stones classifiers. Even if the best results were obtained by the Inception v3 architecture (weighted precision, recall and F1-score of 0.97, 0.98 and 0.97, respectively), it is also shown that choosing an appropriate colour space and texture features allows a shallow machine learning method to approach closely the performances of the most promising deep-learning methods (the XGBoost classifier led to weighted precision, recall and F1-score values of 0.96). This paper is the first one that explores the most discriminant features to be extracted from images acquired during ureteroscopies.