A Rigorous Behavior Assessment of CNNs Using a Data-Domain Sampling Regime
Not on the map. Why?
Abstract (arXiv)
We present a data-domain sampling regime for quantifying CNNs' graphic perception behaviors. This regime lets us evaluate CNNs' ratio estimation ability in bar charts from three perspectives: sensitivity to training-test distribution discrepancies, stability to limited samples, and relative expertise to human observers. After analyzing 16 million trials from 800 CNNs models and 6,825 trials from 113 human participants, we arrived at a simple and actionable conclusion: CNNs can outperform humans and their biases simply depend on the training-test distance. We show evidence of this simple, elegant behavior of the machines when they interpret visualization images. osf.io/gfqc3 provides registration, the code for our sampling regime, and experimental results.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 60 references that are in the map. First posted on arXiv 5 Jul 2025.
Builds on
24 of 60 references.
References · 24
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2016Feature Pyramid Networks for Object Detectioncited by 5,036
- 2015Learning Deep Features for Discriminative Localizationcited by 2,436
- 2020Shortcut Learning in Deep Neural Networkscited by 1,969
- 2016Adversarial examples in the physical worldcited by 1,769
- 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
- 2018Model Cards for Model Reportingcited by 1,080
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
- 2023Evaluating Large Language Models: A Comprehensive Surveycited by 202
- 2017Visualizing Deep Neural Network Decisions: Prediction Difference Analysiscited by 166
12 more
- 2017Perceptual Generative Adversarial Networks for Small Object Detectioncited by 76
- 2020CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualizationcited by 57
- 2016Towards Better Analysis of Deep Convolutional Neural Networkscited by 51
- 2018Visual Analytics for Explainable Deep Learningcited by 39
- 2017Towards Better Analysis of Machine Learning Models: A Visual Analytics Perspectivecited by 36
- 2023Don't trust your eyes: on the (un)reliability of feature visualizationscited by 32
- 2023Visual Analytics For Machine Learning: A Data Perspective Surveycited by 25
- 2025We Can't Understand AI Using our Existing Vocabularycited by 25
- 2023Are Deep Neural Networks Adequate Behavioural Models of Human Visual Perception?cited by 10
- 2024How Aligned are Human Chart Takeaways and LLM Predictions? A Case Study on Bar Charts with Varying Layoutscited by 10
- 2023What Does the Chart Say? Grouping Cues Guide Viewer Comparisons and Conclusions in Bar Chartscited by 3
- 2025Generalization of CNNs on Relational Reasoning with Bar Chartscited by 3
Built on by
Citing papers per year: 2025: 1.
1 paper.