# Optimizing for Generalization in Machine Learning with Cross-Validation Gradients

arXiv: https://arxiv.org/abs/1805.07072 · HTML: https://mlsanity.com/paper/1805.07072/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2018.

## Built on by (most cited first, 2 of 2)

- [Extending the Abstraction of Personality Types based on MBTI with Machine Learning and Natural Language Processing](https://mlsanity.com/paper/2105.11798/) (2021)
- [FedVG: Gradient-Guided Aggregation for Enhanced Federated Learning](https://mlsanity.com/paper/2602.21399/) (2026)

## Abstract (arXiv)

Cross-validation is the workhorse of modern applied statistics and machine learning, as it provides a principled framework for selecting the model that maximizes generalization performance. In this paper, we show that the cross-validation risk is differentiable with respect to the hyperparameters and training data for many common machine learning algorithms, including logistic regression, elastic-net regression, and support vector machines. Leveraging this property of differentiability, we propose a cross-validation gradient method (CVGM) for hyperparameter optimization. Our method enables efficient optimization in high-dimensional hyperparameter spaces of the cross-validation risk, the best surrogate of the true generalization ability of our learning algorithm.