# Invariance-inducing regularization using worst-case transformations suffices to boost accuracy and spatial robustness

arXiv: https://arxiv.org/abs/1906.11235 · HTML: https://mlsanity.com/paper/1906.11235/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2019.

## Built on by (most cited first, 18 of 18)

- [AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation](https://mlsanity.com/paper/2106.04732/) (2021)
- [The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspective](https://mlsanity.com/paper/2210.05021/) (2022)
- [Towards Viewpoint-Invariant Visual Recognition via Adversarial Training](https://mlsanity.com/paper/2307.10235/) (2023)
- [Toward Learning Robust and Invariant Representations with Alignment Regularization and Data Augmentation](https://mlsanity.com/paper/2206.01909/) (2022)
- [Strong inductive biases provably prevent harmless interpolation](https://mlsanity.com/paper/2301.07605/) (2023)
- [Training or Architecture? How to Incorporate Invariance in Neural Networks](https://mlsanity.com/paper/2106.10044/) (2021)
- [Individually Fair Ranking](https://mlsanity.com/paper/2103.11023/) (2021)
- [Improving the Sample-Complexity of Deep Classification Networks with Invariant Integration](https://mlsanity.com/paper/2202.03967/) (2022)
- [Improving Viewpoint Robustness for Visual Recognition via Adversarial Training](https://mlsanity.com/paper/2307.11528/) (2023)
- [Verification of Geometric Robustness of Neural Networks via Piecewise Linear Approximation and Lipschitz Optimisation](https://mlsanity.com/paper/2408.13140/) (2024)
- [Certifying Model Accuracy under Distribution Shifts](https://mlsanity.com/paper/2201.12440/) (2022)
- [Improving Equivariance in State-of-the-Art Supervised Depth and Normal Predictors](https://mlsanity.com/paper/2309.16646/) (2023)
- [A Simple Strategy to Provable Invariance via Orbit Mapping](https://mlsanity.com/paper/2209.11916/) (2022)
- [An Analysis of Causal Effect Estimation using Outcome Invariant Data Augmentation](https://mlsanity.com/paper/2510.25128/) (2025)
- [A Validation Strategy for Deep Learning Models: Evaluating and Enhancing Robustness](https://mlsanity.com/paper/2509.19197/) (2025)
- [Flag Aggregator: Scalable Distributed Training under Failures and Augmented Losses using Convex Optimization](https://mlsanity.com/paper/2302.05865/) (2023)
- [Fine-Grained ImageNet Classification in the Wild](https://mlsanity.com/paper/2303.02400/) (2023)
- [Subspace-Configurable Networks](https://mlsanity.com/paper/2305.13536/) (2023)

## Abstract (arXiv)

This work provides theoretical and empirical evidence that invariance-inducing regularizers can increase predictive accuracy for worst-case spatial transformations (spatial robustness). Evaluated on these adversarially transformed examples, we demonstrate that adding regularization on top of standard or adversarial training reduces the relative error by 20% for CIFAR10 without increasing the computational cost. This outperforms handcrafted networks that were explicitly designed to be spatial-equivariant. Furthermore, we observe for SVHN, known to have inherent variance in orientation, that robust training also improves standard accuracy on the test set. We prove that this no-trade-off phenomenon holds for adversarial examples from transformation groups in the infinite data limit.