# What Teachers Should Know about the Bootstrap: Resampling in the Undergraduate Statistics Curriculum

arXiv: https://arxiv.org/abs/1411.5279 · HTML: https://mlsanity.com/paper/1411.5279/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2014.

## Built on by (most cited first, 15 of 15)

- [Multivariate Anomaly Detection based on Prediction Intervals Constructed using Deep Learning](https://mlsanity.com/paper/2110.03393/) (2021)
- [A Statistics and Deep Learning Hybrid Method for Multivariate Time Series Forecasting and Mortality Modeling](https://mlsanity.com/paper/2112.08618/) (2021)
- [PTEB: Towards Robust Text Embedding Evaluation via Stochastic Paraphrasing at Evaluation Time with LLMs](https://mlsanity.com/paper/2510.06730/) (2025)
- [Causal Falsification of Digital Twins](https://mlsanity.com/paper/2301.07210/) (2023)
- [AI-driven Java Performance Testing: Balancing Result Quality with Testing Time](https://mlsanity.com/paper/2408.05100/) (2024)
- [Bin-Conditional Conformal Prediction of Fatalities from Armed Conflict](https://mlsanity.com/paper/2410.14507/) (2024)
- [Adaptive aggregation of Monte Carlo augmented decomposed filters for efficient group-equivariant convolutional neural network](https://mlsanity.com/paper/2305.10110/) (2023)
- [Model Collapse as Cultural Evolution](https://mlsanity.com/paper/2605.23054/) (2026)
- [The Harder Text Embedding Benchmark (HTEB): Beyond One-dimensional Static Robustness](https://mlsanity.com/paper/2605.28190/) (2026)
- [Can LLM Agents Infer World Models? Evidence from Agentic Automata Learning](https://mlsanity.com/paper/2606.16576/) (2026)
- [Towards Handling Uncertainty-at-Source in AI -- A Review and Next Steps for Interval Regression](https://mlsanity.com/paper/2104.07245/) (2021)
- [Beyond Point Estimates: Distributional Uncertainty in Machine Learning Performance Evaluation](https://mlsanity.com/paper/2501.16931/) (2025)
- [Model Evaluation in the Dark: Robust Classifier Metrics with Missing Labels](https://mlsanity.com/paper/2504.18385/) (2025)
- [On Evaluating and Improving Conversational Agents in Production](https://mlsanity.com/paper/2609.32092/) (2026)
- [How Reliable Are Large Language Models for Soil Organic Carbon Assessment? An Australian Benchmark](https://mlsanity.com/paper/2610.18201/) (2026)

## Abstract (arXiv)

I have three goals in this article: (1) To show the enormous potential of bootstrapping and permutation tests to help students understand statistical concepts including sampling distributions, standard errors, bias, confidence intervals, null distributions, and P-values. (2) To dig deeper, understand why these methods work and when they don't, things to watch out for, and how to deal with these issues when teaching. (3) To change statistical practice---by comparing these methods to common t tests and intervals, we see how inaccurate the latter are; we confirm this with asymptotics. n >= 30 isn't enough---think n >= 5000. Resampling provides diagnostics, and more accurate alternatives. Sadly, the common bootstrap percentile interval badly under-covers in small samples; there are better alternatives. The tone is informal, with a few stories and jokes.