# A Rigorous Behavior Assessment of CNNs Using a Data-Domain Sampling Regime

arXiv: https://arxiv.org/abs/2507.03866 · HTML: https://mlsanity.com/paper/2507.03866/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 60 references that are in the map.
- First posted on arXiv 5 Jul 2025.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces](https://mlsanity.com/paper/2412.14171/) (2024)
- [Evaluating Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2310.19736/) (2023)
- [Visualizing Deep Neural Network Decisions: Prediction Difference Analysis](https://mlsanity.com/paper/1702.04595/) (2017)
- [Perceptual Generative Adversarial Networks for Small Object Detection](https://mlsanity.com/paper/1706.05274/) (2017)
- [CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization](https://mlsanity.com/paper/2004.15004/) (2020)
- [Towards Better Analysis of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1604.07043/) (2016)
- [Visual Analytics for Explainable Deep Learning](https://mlsanity.com/paper/1804.02527/) (2018)
- [Towards Better Analysis of Machine Learning Models: A Visual Analytics Perspective](https://mlsanity.com/paper/1702.01226/) (2017)
- [Don't trust your eyes: on the (un)reliability of feature visualizations](https://mlsanity.com/paper/2306.04719/) (2023)
- [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023)
- [We Can't Understand AI Using our Existing Vocabulary](https://mlsanity.com/paper/2502.07586/) (2025)
- [Are Deep Neural Networks Adequate Behavioural Models of Human Visual Perception?](https://mlsanity.com/paper/2305.17023/) (2023)
- [How Aligned are Human Chart Takeaways and LLM Predictions? A Case Study on Bar Charts with Varying Layouts](https://mlsanity.com/paper/2408.06837/) (2024)
- [What Does the Chart Say? Grouping Cues Guide Viewer Comparisons and Conclusions in Bar Charts](https://mlsanity.com/paper/2310.02076/) (2023)
- [Generalization of CNNs on Relational Reasoning with Bar Charts](https://mlsanity.com/paper/2503.00086/) (2025)

## Built on by (most cited first, 1 of 1)

- [GPT-5 Model Corrected GPT-4V's Chart Reading Errors, Not Prompting](https://mlsanity.com/paper/2510.06782/) (2025)

## Abstract (arXiv)

We present a data-domain sampling regime for quantifying CNNs' graphic perception behaviors. This regime lets us evaluate CNNs' ratio estimation ability in bar charts from three perspectives: sensitivity to training-test distribution discrepancies, stability to limited samples, and relative expertise to human observers. After analyzing 16 million trials from 800 CNNs models and 6,825 trials from 113 human participants, we arrived at a simple and actionable conclusion: CNNs can outperform humans and their biases simply depend on the training-test distance. We show evidence of this simple, elegant behavior of the machines when they interpret visualization images. osf.io/gfqc3 provides registration, the code for our sampling regime, and experimental results.