# Model Selection Techniques -- An Overview

arXiv: https://arxiv.org/abs/1810.09583 · HTML: https://mlsanity.com/paper/1810.09583/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2018.

## Built on by (most cited first, 30 of 36)

- [Transferability Estimation using Bhattacharyya Class Separability](https://mlsanity.com/paper/2111.12780/) (2021)
- [Pruning Deep Neural Networks from a Sparsity Perspective](https://mlsanity.com/paper/2302.05601/) (2023)
- [MAP: Multi-Human-Value Alignment Palette](https://mlsanity.com/paper/2410.19198/) (2024)
- [Federated Learning Challenges and Opportunities: An Outlook](https://mlsanity.com/paper/2202.00807/) (2022)
- [Trust in AI: Interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficient](https://mlsanity.com/paper/2202.05302/) (2022)
- [Demystifying Poisoning Backdoor Attacks from a Statistical Perspective](https://mlsanity.com/paper/2310.10780/) (2023)
- [Model Reporting for Certifiable AI: A Proposal from Merging EU Regulation into AI Development](https://mlsanity.com/paper/2307.11525/) (2023)
- [A Priori Denoising Strategies for Sparse Identification of Nonlinear Dynamical Systems: A Comparative Study](https://mlsanity.com/paper/2201.12683/) (2022)
- [Disentangling Model Multiplicity in Deep Learning](https://mlsanity.com/paper/2206.08890/) (2022)
- [Evaluation of Question Answering Systems: Complexity of judging a natural language](https://mlsanity.com/paper/2209.12617/) (2022)
- [The Rate of Convergence of Variation-Constrained Deep Neural Networks](https://mlsanity.com/paper/2106.12068/) (2021)
- [Provable Identifiability of Two-Layer ReLU Neural Networks via LASSO Regularization](https://mlsanity.com/paper/2305.04267/) (2023)
- [Detecting Signs of Model Change with Continuous Model Selection Based on Descriptive Dimensionality](https://mlsanity.com/paper/2302.12127/) (2023)
- [Stabilizing black-box model selection with the inflated argmax](https://mlsanity.com/paper/2410.18268/) (2024)
- [Assisted Learning for Organizations with Limited Imbalanced Data](https://mlsanity.com/paper/2109.09307/) (2021)
- [Detection and Evaluation of Clusters within Sequential Data](https://mlsanity.com/paper/2210.01679/) (2022)
- [To tree or not to tree? Assessing the impact of smoothing the decision boundaries](https://mlsanity.com/paper/2210.03672/) (2022)
- [Challenges and Opportunities of Using Transformer-Based Multi-Task Learning in NLP Through ML Lifecycle: A Survey](https://mlsanity.com/paper/2308.08234/) (2023)
- [Online Estimation with Rolling Validation: Adaptive Nonparametric Estimation with Streaming Data](https://mlsanity.com/paper/2310.12140/) (2023)
- [Policy Trees for Prediction: Interpretable and Adaptive Model Selection for Machine Learning](https://mlsanity.com/paper/2405.20486/) (2024)
- [Model Privacy: A Unified Framework for Understanding Model Stealing Attacks and Defenses](https://mlsanity.com/paper/2502.15567/) (2025)
- [Evidence-Driven Decision Support for AI Model Selection in Research Software Engineering](https://mlsanity.com/paper/2512.11984/) (2025)
- [Proteus: Incremental Memory Activation for Long-Context Sequence Modeling](https://mlsanity.com/paper/2608.16844/) (2026)
- [A hybrid ensemble method with negative correlation learning for regression](https://mlsanity.com/paper/2104.02317/) (2021)
- [Learning the hypotheses space from data through a U-curve algorithm](https://mlsanity.com/paper/2109.03866/) (2021)
- [Targeted Cross-Validation](https://mlsanity.com/paper/2109.06949/) (2021)
- [Unifying Epidemic Models with Mixtures](https://mlsanity.com/paper/2201.04960/) (2022)
- [Diagnostic Tool for Out-of-Sample Model Evaluation](https://mlsanity.com/paper/2206.10982/) (2022)
- [Exact identification of nonlinear dynamical systems by Trimmed Lasso](https://mlsanity.com/paper/2308.01891/) (2023)
- [Behavior Matters: An Alternative Perspective on Promoting Responsible Data Science](https://mlsanity.com/paper/2410.17273/) (2024)

## Abstract (arXiv)

In the era of big data, analysts usually explore various statistical models or machine learning methods for observed data in order to facilitate scientific discoveries or gain predictive power. Whatever data and fitting procedures are employed, a crucial step is to select the most appropriate model or method from a set of candidates. Model selection is a key ingredient in data analysis for reliable and reproducible statistical inference or prediction, and thus central to scientific studies in fields such as ecology, economics, engineering, finance, political science, biology, and epidemiology. There has been a long history of model selection techniques that arise from researches in statistics, information theory, and signal processing. A considerable number of methods have been proposed, following different philosophies and exhibiting varying performances. The purpose of this article is to bring a comprehensive overview of them, in terms of their motivation, large sample performance, and applicability. We provide integrated and practically relevant discussions on theoretical properties of state-of- the-art model selection approaches. We also share our thoughts on some controversial views on the practice of model selection.