# Identifying and Exploiting Structures for Reliable Deep Learning

arXiv: https://arxiv.org/abs/2108.07083 · HTML: https://mlsanity.com/paper/2108.07083/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 298 references that are in the map.
- First posted on arXiv 16 Aug 2021.

## Builds on (most cited first, 23 of 23)

- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Surprises in High-Dimensional Ridgeless Least Squares Interpolation](https://mlsanity.com/paper/1903.08560/) (2019)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [Differential Privacy Has Disparate Impact on Model Accuracy](https://mlsanity.com/paper/1905.12101/) (2019)
- [Geometric GAN](https://mlsanity.com/paper/1705.02894/) (2017)
- [Spectral Norm Regularization for Improving the Generalizability of Deep Learning](https://mlsanity.com/paper/1705.10941/) (2017)
- [Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption](https://mlsanity.com/paper/1711.10677/) (2017)
- [What Do Compressed Deep Neural Networks Forget?](https://mlsanity.com/paper/1911.05248/) (2019)
- [Adversarial Training Can Hurt Generalization](https://mlsanity.com/paper/1906.06032/) (2019)
- [Local Temperature Scaling for Probability Calibration](https://mlsanity.com/paper/2008.05105/) (2020)
- [Finite-sample Analysis of Interpolating Linear Classifiers in the Overparameterized Regime](https://mlsanity.com/paper/2004.12019/) (2020)
- [Chiron: Privacy-preserving Machine Learning as a Service](https://mlsanity.com/paper/1803.05961/) (2018)
- [FANet: A Feedback Attention Network for Improved Biomedical Image Segmentation](https://mlsanity.com/paper/2103.17235/) (2021)
- [GAT: Generative Adversarial Training for Adversarial Example Detection and Robust Classification](https://mlsanity.com/paper/1905.11475/) (2019)
- [Robust Convolutional Neural Networks under Adversarial Noise](https://mlsanity.com/paper/1511.06306/) (2015)
- [Adversarial Examples from Cryptographic Pseudo-Random Generators](https://mlsanity.com/paper/1811.06418/) (2018)
- [Adversarially Robust Training through Structured Gradient Regularization](https://mlsanity.com/paper/1805.08736/) (2018)
- [Adversarial Robustness via Label-Smoothing](https://mlsanity.com/paper/1906.11567/) (2019)
- [Hidden Cost of Randomized Smoothing](https://mlsanity.com/paper/2003.01249/) (2020)
- [Query complexity of adversarial attacks](https://mlsanity.com/paper/2010.01039/) (2020)

## Abstract (arXiv)

Deep learning research has recently witnessed an impressively fast-paced progress in a wide range of tasks including computer vision, natural language processing, and reinforcement learning. The extraordinary performance of these systems often gives the impression that they can be used to revolutionise our lives for the better. However, as recent works point out, these systems suffer from several issues that make them unreliable for use in the real world, including vulnerability to adversarial attacks (Szegedy et al. [248]), tendency to memorise noise (Zhang et al. [292]), being over-confident on incorrect predictions (miscalibration) (Guo et al. [99]), and unsuitability for handling private data (Gilad-Bachrach et al. [88]). In this thesis, we look at each of these issues in detail, investigate their causes, and propose computationally cheap algorithms for mitigating them in practice. To do this, we identify structures in deep neural networks that can be exploited to mitigate the above causes of unreliability of deep learning algorithms.