# I Am Big, You Are Little; I Am Right, You Are Wrong

arXiv: https://arxiv.org/abs/2507.23509 · HTML: https://mlsanity.com/paper/2507.23509/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 42 references that are in the map.
- First posted on arXiv 31 Jul 2025.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Models](https://mlsanity.com/paper/1808.01688/) (2018)
- [Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels](https://mlsanity.com/paper/2101.05022/) (2021)
- [When does dough become a bagel? Analyzing the remaining mistakes on ImageNet](https://mlsanity.com/paper/2205.04596/) (2022)
- [One Explanation is Not Enough: Structured Attention Graphs for Image Classification](https://mlsanity.com/paper/2011.06733/) (2020)
- [Comparing the Decision-Making Mechanisms by Transformers and CNNs via Explanation Methods](https://mlsanity.com/paper/2212.06872/) (2022)
- [Flaws of ImageNet, Computer Vision's Favourite Dataset](https://mlsanity.com/paper/2412.00076/) (2024)
- [Explaining Image Classifiers](https://mlsanity.com/paper/2401.13752/) (2024)
- [MRxaI: Black-Box Explainability for Image Classifiers in a Medical Setting](https://mlsanity.com/paper/2311.14471/) (2023)
- [White Box Methods for Explanations of Convolutional Neural Networks in Image Classification Tasks](https://mlsanity.com/paper/2104.02548/) (2021)

## Built on by (most cited first, 3 of 3)

- [Out-of-the-box: Black-box Causal Attacks on Object Detectors](https://mlsanity.com/paper/2512.03730/) (2025)
- [I Guess That's Why They Call it the Blues: Causal Analysis for Audio Classifiers](https://mlsanity.com/paper/2601.16675/) (2026)
- [Activation-Deactivation: A General Framework for Robust Post-hoc Explainable AI](https://mlsanity.com/paper/2510.01038/) (2025)

## Abstract (arXiv)

Machine learning for image classification is an active and rapidly developing field. With the proliferation of classifiers of different sizes and different architectures, the problem of choosing the right model becomes more and more important. While we can assess a model's classification accuracy statistically, our understanding of the way these models work is unfortunately limited. In order to gain insight into the decision-making process of different vision models, we propose using minimal sufficient pixels sets to gauge a model's `concentration': the pixels that capture the essence of an image through the lens of the model. By comparing position, overlap, and size of sets of pixels, we identify that different architectures have statistically different concentration, in both size and position. In particular, ConvNext and EVA models differ markedly from the others. We also identify that images which are misclassified are associated with larger pixels sets than correct classifications.