# Valid Inference for Machine Learning Model Parameters

arXiv: https://arxiv.org/abs/2302.10840 · HTML: https://mlsanity.com/paper/2302.10840/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 52 references that are in the map.
- First posted on arXiv 21 Feb 2023.

## Builds on (most cited first, 12 of 12)

- [Estimating means of bounded random variables by betting](https://mlsanity.com/paper/2010.09686/) (2020)
- [Game-theoretic statistics and safe anytime-valid inference](https://mlsanity.com/paper/2210.01948/) (2022)
- [Safe Testing](https://mlsanity.com/paper/1906.07801/) (2019)
- [Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It](https://mlsanity.com/paper/1412.3730/) (2014)
- [Universal Inference](https://mlsanity.com/paper/1912.11436/) (2019)
- [Fast Rates for General Unbounded Loss Functions: from ERM to Generalized Bayes](https://mlsanity.com/paper/1605.00252/) (2016)
- [A comparison of learning rate selection methods in generalized Bayesian inference](https://mlsanity.com/paper/2012.11349/) (2020)
- [Beyond Neyman-Pearson: e-values enable hypothesis testing with a data-driven alpha](https://mlsanity.com/paper/2205.00901/) (2022)
- [Inference on function-valued parameters using a restricted score test](https://mlsanity.com/paper/2105.06646/) (2021)
- [Gibbs posterior inference on multivariate quantiles](https://mlsanity.com/paper/2002.01052/) (2020)
- [Direct and approximately valid probabilistic inference on a class of statistical functionals](https://mlsanity.com/paper/2112.10232/) (2021)
- [Statistical Inference for Bayesian Risk Minimization via Exponentially Tilted Empirical Likelihood](https://mlsanity.com/paper/2109.07792/) (2021)

## Built on by (most cited first, 1 of 1)

- [Model-free generalized fiducial inference](https://mlsanity.com/paper/2307.12472/) (2023)

## Abstract (arXiv)

The parameters of a machine learning model are typically learned by minimizing a loss function on a set of training data. However, this can come with the risk of overtraining; in order for the model to generalize well, it is of great importance that we are able to find the optimal parameter for the model on the entire population -- not only on the given training sample. In this paper, we construct valid confidence sets for this optimal parameter of a machine learning model, which can be generated using only the training data without any knowledge of the population. We then show that studying the distribution of this confidence set allows us to assign a notion of confidence to arbitrary regions of the parameter space, and we demonstrate that this distribution can be well-approximated using bootstrapping techniques.