# Generalizability of Machine Learning Models: Quantitative Evaluation of Three Methodological Pitfalls

arXiv: https://arxiv.org/abs/2202.01337 · HTML: https://mlsanity.com/paper/2202.01337/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 57 references that are in the map.
- First posted on arXiv 1 Feb 2022.

## Builds on (most cited first, 9 of 9)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging](https://mlsanity.com/paper/1909.12475/) (2019)
- [Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans](https://mlsanity.com/paper/2008.06388/) (2020)
- [Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks](https://mlsanity.com/paper/1901.11489/) (2019)
- [Improved Segmentation and Detection Sensitivity of Diffusion-Weighted Brain Infarct Lesions with Synthetically Enhanced Deep Learning](https://mlsanity.com/paper/2012.14771/) (2020)

## Built on by (most cited first, 12 of 12)

- [Breast Cancer Classification with Enhanced Interpretability: DALAResNet50 and DT Grad-CAM](https://mlsanity.com/paper/2308.13150/) (2023)
- [A-Eval: A Benchmark for Cross-Dataset Evaluation of Abdominal Multi-Organ Segmentation](https://mlsanity.com/paper/2309.03906/) (2023)
- [Learn2Synth: Learning Optimal Data Synthesis Using Hypergradients for Brain Image Segmentation](https://mlsanity.com/paper/2411.16719/) (2024)
- [Beyond Real Data: Synthetic Data through the Lens of Regularization](https://mlsanity.com/paper/2510.08095/) (2025)
- [Building Trustworthy AI for Materials Discovery: From Autonomous Laboratories to Z-scores](https://mlsanity.com/paper/2512.01080/) (2025)
- [M-Loss: Quantifying Model Merging Compatibility with Limited Unlabeled Data](https://mlsanity.com/paper/2602.08564/) (2026)
- [Picking the Right Specialist: Attentive Neural Process-based Selection of Task-Specialized Models as Tools for Agentic Healthcare Systems](https://mlsanity.com/paper/2602.14901/) (2026)
- [Statistical Quality and Reproducibility of Pseudorandom Number Generators in Machine Learning technologies](https://mlsanity.com/paper/2507.03007/) (2025)
- [RIDGE: Reproducibility, Integrity, Dependability, Generalizability, and Efficiency Assessment of Medical Image Segmentation Models](https://mlsanity.com/paper/2401.08847/) (2024)
- [At the junction between deep learning and statistics of extremes: formalizing the landslide hazard definition](https://mlsanity.com/paper/2401.14210/) (2024)
- [Towards physics-informed neural networks for landslide prediction](https://mlsanity.com/paper/2407.06785/) (2024)
- [Exploration and Evaluation of Bias in Cyberbullying Detection with Machine Learning](https://mlsanity.com/paper/2412.00609/) (2024)

## Abstract (arXiv)

Purpose: Despite the potential of machine learning models, the lack of generalizability has hindered their widespread adoption in clinical practice. We investigate three methodological pitfalls: (1) violation of independence assumption, (2) model evaluation with an inappropriate performance indicator or baseline for comparison, and (3) batch effect. Materials and Methods: Using several retrospective datasets, we implement machine learning models with and without the pitfalls to quantitatively illustrate these pitfalls' effect on model generalizability. Results: Violation of independence assumption, more specifically, applying oversampling, feature selection, and data augmentation before splitting data into train, validation, and test sets, respectively, led to misleading and superficial gains in F1 scores of 71.2% in predicting local recurrence and 5.0% in predicting 3-year overall survival in head and neck cancer as well as 46.0% in distinguishing histopathological patterns in lung cancer. Further, randomly distributing data points for a subject across training, validation, and test sets led to a 21.8% superficial increase in F1 score. Also, we showed the importance of the choice of performance measures and baseline for comparison. In the presence of batch effect, a model built for pneumonia detection led to F1 score of 98.7%. However, when the same model was applied to a new dataset of normal patients, it only correctly classified 3.86% of the samples. Conclusions: These methodological pitfalls cannot be captured using internal model evaluation, and the inaccurate predictions made by such models may lead to wrong conclusions and interpretations. Therefore, understanding and avoiding these pitfalls is necessary for developing generalizable models.