# Fast Differentiable Clipping-Aware Normalization and Rescaling

arXiv: https://arxiv.org/abs/2007.07677 · HTML: https://mlsanity.com/paper/2007.07677/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2020.

## Built on by (most cited first, 5 of 5)

- [Smart App Attack: Hacking Deep Learning Models in Android Apps](https://mlsanity.com/paper/2204.11075/) (2022)
- [Targeted Adversarial Attacks on Wind Power Forecasts](https://mlsanity.com/paper/2303.16633/) (2023)
- [DDDM: a Brain-Inspired Framework for Robust Classification](https://mlsanity.com/paper/2205.10117/) (2022)
- [Denoised Internal Models: a Brain-Inspired Autoencoder against Adversarial Attacks](https://mlsanity.com/paper/2111.10844/) (2021)
- [Learning Representations Robust to Group Shifts and Adversarial Examples](https://mlsanity.com/paper/2202.09446/) (2022)

## Abstract (arXiv)

Rescaling a vector vecδ ∈ ℝn to a desired length is a common operation in many areas such as data science and machine learning. When the rescaled perturbation η vecδ is added to a starting point vecx ∈ D (where D is the data domain, e.g. D = [0, 1]n), the resulting vector vecv = vecx + η vecδ will in general not be in D. To enforce that the perturbed vector v is in D, the values of vecv can be clipped to D. This subsequent element-wise clipping to the data domain does however reduce the effective perturbation size and thus interferes with the rescaling of vecδ. The optimal rescaling η to obtain a perturbation with the desired norm after the clipping can be iteratively approximated using a binary search. However, such an iterative approach is slow and non-differentiable. Here we show that the optimal rescaling can be found analytically using a fast and differentiable algorithm. Our algorithm works for any p-norm and can be used to train neural networks on inputs with normalized perturbations. We provide native implementations for PyTorch, TensorFlow, JAX, and NumPy based on EagerPy.