# A Statistician Teaches Deep Learning

arXiv: https://arxiv.org/abs/2102.01194 · HTML: https://mlsanity.com/paper/2102.01194/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 103 references that are in the map.
- First posted on arXiv 29 Jan 2021.

## Builds on (most cited first, 30 of 32)

- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [Deep Double Descent: Where Bigger Models and More Data Hurt](https://mlsanity.com/paper/1912.02292/) (2019)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [Size-Independent Sample Complexity of Neural Networks](https://mlsanity.com/paper/1712.06541/) (2017)
- [Scalable and accurate deep learning for electronic health records](https://mlsanity.com/paper/1801.07860/) (2018)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [High-dimensional dynamics of generalization error in neural networks](https://mlsanity.com/paper/1710.03667/) (2017)
- [Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models](https://mlsanity.com/paper/1708.08296/) (2017)
- [Optimal Regularization Can Mitigate Double Descent](https://mlsanity.com/paper/2003.01897/) (2020)
- [Deep Learning for Medical Image Processing: Overview, Challenges and Future](https://mlsanity.com/paper/1704.06825/) (2017)
- [SparseFool: a few pixels make a big difference](https://mlsanity.com/paper/1811.02248/) (2018)
- [Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification](https://mlsanity.com/paper/1802.00850/) (2018)
- [The jamming transition as a paradigm to understand the loss landscape of deep neural networks](https://mlsanity.com/paper/1809.09349/) (2018)
- [Compression of Neural Machine Translation Models via Pruning](https://mlsanity.com/paper/1606.09274/) (2016)
- [An Introduction to Image Synthesis with Generative Adversarial Nets](https://mlsanity.com/paper/1803.04469/) (2018)
- [Deep Neural Networks to Enable Real-time Multimessenger Astrophysics](https://mlsanity.com/paper/1701.00008/) (2016)
- [Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models](https://mlsanity.com/paper/1604.00788/) (2016)
- [Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey](https://mlsanity.com/paper/1512.03131/) (2015)
- [Bridging Maximum Likelihood and Adversarial Learning via α-Divergence](https://mlsanity.com/paper/2007.06178/) (2020)
- [A Survey on Deep Learning Toolkits and Libraries for Intelligent User Interfaces](https://mlsanity.com/paper/1803.04818/) (2018)

## Abstract (arXiv)

Deep learning (DL) has gained much attention and become increasingly popular in modern data science. Computer scientists led the way in developing deep learning techniques, so the ideas and perspectives can seem alien to statisticians. Nonetheless, it is important that statisticians become involved -- many of our students need this expertise for their careers. In this paper, developed as part of a program on DL held at the Statistical and Applied Mathematical Sciences Institute, we address this culture gap and provide tips on how to teach deep learning to statistics graduate students. After some background, we list ways in which DL and statistical perspectives differ, provide a recommended syllabus that evolved from teaching two iterations of a DL graduate course, offer examples of suggested homework assignments, give an annotated list of teaching resources, and discuss DL in the context of two research areas.