# Stretching Beyond the Obvious: A Gradient-Free Framework to Unveil the Hidden Landscape of Visual Invariance

arXiv: https://arxiv.org/abs/2506.17040 · HTML: https://mlsanity.com/paper/2506.17040/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 82 references that are in the map.
- First posted on arXiv 20 Jun 2025.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [Understanding Deep Image Representations by Inverting Them](https://mlsanity.com/paper/1412.0035/) (2014)
- [Do Adversarially Robust ImageNet Models Transfer Better?](https://mlsanity.com/paper/2007.08489/) (2020)
- [Intrinsic dimension of data representations in deep neural networks](https://mlsanity.com/paper/1905.12784/) (2019)
- [Estimating the intrinsic dimension of datasets by a minimal neighborhood information](https://mlsanity.com/paper/1803.06992/) (2018)
- [Adversarial Audio Synthesis](https://mlsanity.com/paper/1802.04208/) (2018)
- [Predicting Parameters in Deep Learning](https://mlsanity.com/paper/1306.0543/) (2013)
- [Generating Images with Perceptual Similarity Metrics based on Deep Networks](https://mlsanity.com/paper/1602.02644/) (2016)
- [Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks](https://mlsanity.com/paper/1602.03616/) (2016)
- [A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking](https://mlsanity.com/paper/2302.14301/) (2023)
- [Excessive Invariance Causes Adversarial Vulnerability](https://mlsanity.com/paper/1811.00401/) (2018)
- [Understanding Neural Networks via Feature Visualization: A survey](https://mlsanity.com/paper/1904.08939/) (2019)
- [Harmonizing the object recognition strategies of deep neural networks with humans](https://mlsanity.com/paper/2211.04533/) (2022)
- [Unlocking Feature Visualization for Deeper Networks with MAgnitude Constrained Optimization](https://mlsanity.com/paper/2306.06805/) (2023)
- [Eigen-Distortions of Hierarchical Representations](https://mlsanity.com/paper/1710.02266/) (2017)
- [Analyzing biological and artificial neural networks: challenges with opportunities for synergy?](https://mlsanity.com/paper/1810.13373/) (2018)
- [Deep Networks Can Resemble Human Feed-forward Vision in Invariant Object Recognition](https://mlsanity.com/paper/1508.03929/) (2015)
- [Geodesics of learned representations](https://mlsanity.com/paper/1511.06394/) (2015)
- [Diverse feature visualizations reveal invariances in early layers of deep neural networks](https://mlsanity.com/paper/1807.10589/) (2018)
- [Gradient-free activation maximization for identifying effective stimuli](https://mlsanity.com/paper/1905.00378/) (2019)
- [High-performance Evolutionary Algorithms for Online Neuron Control](https://mlsanity.com/paper/2204.06765/) (2022)
- [Extreme Image Transformations Affect Humans and Machines Differently](https://mlsanity.com/paper/2212.13967/) (2022)
- [Prune and distill: similar reformatting of image information along rat visual cortex and deep neural networks](https://mlsanity.com/paper/2205.13816/) (2022)
- [L-WISE: Boosting Human Visual Category Learning Through Model-Based Image Selection and Enhancement](https://mlsanity.com/paper/2412.09765/) (2024)

## Abstract (arXiv)

Uncovering which feature combinations are encoded by visual units is critical to understanding how images are transformed into representations that support recognition. While existing feature visualization approaches typically infer a unit's most exciting images, this is insufficient to reveal the manifold of transformations under which responses remain invariant, which is critical to generalization in vision. Here we introduce Stretch-and-Squeeze (SnS), a model-agnostic, gradient-free framework to systematically characterize a unit's maximally invariant stimuli, and its vulnerability to adversarial perturbations, in both biological and artificial visual systems. SnS frames these transformations as bi-objective optimization problems. To probe invariance, SnS seeks image perturbations that maximally alter (stretch) the representation of a reference stimulus in a given processing stage while preserving unit activation downstream (squeeze). To probe adversarial sensitivity, stretching and squeezing are reversed to maximally perturb unit activation while minimizing changes to the upstream representation. Applied to CNNs, SnS revealed invariant transformations that were farther from a reference image in pixel-space than those produced by affine transformations, while more strongly preserving the target unit's response. The discovered invariant images differed depending on the stage of the image representation used for optimization: pixel-level changes primarily affected luminance and contrast, while stretching mid- and late-layer representations mainly altered texture and pose. By measuring how well the hierarchical invariant images obtained for L2 robust networks were classified by humans and other observer networks, we discovered a substantial drop in their interpretability when the representation was stretched in deep layers, while the opposite trend was found for standard models.