# Generalization to translation shifts: a study in architectures and augmentations

arXiv: https://arxiv.org/abs/2207.02349 · HTML: https://mlsanity.com/paper/2207.02349/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 49 references that are in the map.
- First posted on arXiv 5 Jul 2022.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [Going deeper with Image Transformers](https://mlsanity.com/paper/2103.17239/) (2021)
- [How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers](https://mlsanity.com/paper/2106.10270/) (2021)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [ResMLP: Feedforward networks for image classification with data-efficient training](https://mlsanity.com/paper/2105.03404/) (2021)
- [When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations](https://mlsanity.com/paper/2106.01548/) (2021)
- [Understanding Robustness of Transformers for Image Classification](https://mlsanity.com/paper/2103.14586/) (2021)
- [Vision Transformers are Robust Learners](https://mlsanity.com/paper/2105.07581/) (2021)
- [Why do deep convolutional networks generalize so poorly to small image transformations?](https://mlsanity.com/paper/1805.12177/) (2018)
- [Spatially Transformed Adversarial Examples](https://mlsanity.com/paper/1801.02612/) (2018)
- [Are Transformers More Robust Than CNNs?](https://mlsanity.com/paper/2111.05464/) (2021)
- [On the Robustness of Vision Transformers to Adversarial Examples](https://mlsanity.com/paper/2104.02610/) (2021)
- [On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Location](https://mlsanity.com/paper/2003.07064/) (2020)
- [On Robustness and Transferability of Convolutional Neural Networks](https://mlsanity.com/paper/2007.08558/) (2020)
- [Data Augmentation by Pairing Samples for Images Classification](https://mlsanity.com/paper/1801.02929/) (2018)
- [Truly shift-invariant convolutional neural networks](https://mlsanity.com/paper/2011.14214/) (2020)
- [Micro-Batch Training with Batch-Channel Normalization and Weight Standardization](https://mlsanity.com/paper/1903.10520/) (2019)

## Built on by (most cited first, 3 of 3)

- [Alias-Free Convnets: Fractional Shift Invariance via Polynomial Activations](https://mlsanity.com/paper/2303.08085/) (2023)
- [Alias-Free ViT: Fractional Shift Invariance via Linear Attention](https://mlsanity.com/paper/2510.22673/) (2025)
- [Lost in Translation: Modern Neural Networks Still Struggle With Small Realistic Image Transformations](https://mlsanity.com/paper/2404.07153/) (2024)

## Abstract (arXiv)

We study how effective data augmentation is at capturing the inductive bias of carefully designed network architectures for spatial translation invariance. We evaluate various image classification architectures (antialiased, convolutional, vision transformer, and fully connected MLP networks) and data augmentation techniques towards generalization to large translation shifts. We observe that: (a) without data augmentation, all architectures, including convolutional networks with antialiased modification suffer some degradation in performance when evaluated on translated test distributions. Understandably, both the in-distribution accuracy and degradation to shifts is significantly worse for non-convolutional models. (b) The robustness of performance is improved by even a minimal augmentation of 4 pixel random crop across all architectures. In some instances, even 1-2 pixel random crop is sufficient. This suggests that there is a form of meta generalization from augmentation. For non-convolutional architectures, while the absolute accuracy is still low with this basic augmentation, we see substantial improvements in robustness to translation shifts. (c) With a sufficiently advanced augmentation pipeline (4 pixel crop+RandAugmentation+Erasing+MixUp), all architectures can be trained to have competitive performance in terms of in-distribution accuracy as well as generalization to large translation shifts.