# Towards Robust, Locally Linear Deep Networks

arXiv: https://arxiv.org/abs/1907.03207 · HTML: https://mlsanity.com/paper/1907.03207/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jul 2019.

## Built on by (most cited first, 20 of 20)

- [Relating Adversarially Robust Generalization to Flat Minima](https://mlsanity.com/paper/2104.04448/) (2021)
- [Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions](https://mlsanity.com/paper/2108.09108/) (2021)
- [Deep Constrained Least Squares for Blind Image Super-Resolution](https://mlsanity.com/paper/2202.07508/) (2022)
- [When Deep Learning Meets Polyhedral Theory: A Survey](https://mlsanity.com/paper/2305.00241/) (2023)
- [DWDN: Deep Wiener Deconvolution Network for Non-Blind Image Deblurring](https://mlsanity.com/paper/2103.09962/) (2021)
- [Neural Response Interpretation through the Lens of Critical Pathways](https://mlsanity.com/paper/2103.16886/) (2021)
- [What Makes a Good Explanation?: A Harmonized View of Properties of Explanations](https://mlsanity.com/paper/2211.05667/) (2022)
- [How and When Adversarial Robustness Transfers in Knowledge Distillation?](https://mlsanity.com/paper/2110.12072/) (2021)
- [On the Expected Complexity of Maxout Networks](https://mlsanity.com/paper/2107.00379/) (2021)
- [Sound and Complete Neural Network Repair with Minimality and Locality Guarantees](https://mlsanity.com/paper/2110.07682/) (2021)
- [Recent Advances in Large Margin Learning](https://mlsanity.com/paper/2103.13598/) (2021)
- [What training reveals about neural network complexity](https://mlsanity.com/paper/2106.04186/) (2021)
- [Reverse Engineering ℓp attacks: A block-sparse optimization approach with recovery guarantees](https://mlsanity.com/paper/2203.04886/) (2022)
- [F2former: When Fractional Fourier Meets Deep Wiener Deconvolution and Selective Frequency Transformer for Image Deblurring](https://mlsanity.com/paper/2409.02056/) (2024)
- [Learning Prescriptive ReLU Networks](https://mlsanity.com/paper/2306.00651/) (2023)
- [SurvReLU: Inherently Interpretable Survival Analysis via Deep ReLU Networks](https://mlsanity.com/paper/2407.14463/) (2024)
- [S-CFE: Simple Counterfactual Explanations](https://mlsanity.com/paper/2410.15723/) (2024)
- [SpecXAI -- Spectral interpretability of Deep Learning Models](https://mlsanity.com/paper/2302.09949/) (2023)
- [Instance-wise Linearization of Neural Network for Model Interpretation](https://mlsanity.com/paper/2310.16295/) (2023)
- [PRUNE: A Patching Based Repair Framework for Certifiable Unlearning of Neural Networks](https://mlsanity.com/paper/2505.06520/) (2025)

## Abstract (arXiv)

Deep networks realize complex mappings that are often understood by their locally linear behavior at or around points of interest. For example, we use the derivative of the mapping with respect to its inputs for sensitivity analysis, or to explain (obtain coordinate relevance for) a prediction. One key challenge is that such derivatives are themselves inherently unstable. In this paper, we propose a new learning problem to encourage deep networks to have stable derivatives over larger regions. While the problem is challenging in general, we focus on networks with piecewise linear activation functions. Our algorithm consists of an inference step that identifies a region around a point where linear approximation is provably stable, and an optimization step to expand such regions. We propose a novel relaxation to scale the algorithm to realistic models. We illustrate our method with residual and recurrent networks on image and sequence datasets.