mlsanityData to 14 Oct 2026

Seeing Eye to AI? Applying Deep-Feature-Based Similarity Metrics to Information Visualization

arXiv 2503.00228 ↗First posted 28 Feb 2025cs.HC cs.CV cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Judging the similarity of visualizations is crucial to various applications, such as visualization-based search and visualization recommendation systems. Recent studies show deep-feature-based similarity metrics correlate well with perceptual judgments of image similarity and serve as effective loss functions for tasks like image super-resolution and style transfer. We explore the application of such metrics to judgments of visualization similarity. We extend a similarity metric using five ML architectures and three pre-trained weight sets. We replicate results from previous crowd-sourced studies on scatterplot and visual channel similarity perception. Notably, our metric using pre-trained ImageNet weights outperformed gradient-descent tuned MS-SSIM, a multi-scale similarity metric based on luminance, contrast, and structure. Our work contributes to understanding how deep-feature-based metrics can enhance similarity assessments in visualization, potentially improving visual analysis tools and techniques. Supplementary materials are available at https://osf.io/dj2ms.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 31 of its 91 references that are in the map. First posted on arXiv 28 Feb 2025.

Builds on

31 of 91 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  8. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  9. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  10. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  11. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  12. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
19 more
  1. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
  2. 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 811
  3. 2015A Neural Algorithm of Artistic Stylecited by 483
  4. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
  5. 2018PieAPP: Perceptual Image-Error Assessment through Pairwise Preferencecited by 160
  6. 2023Getting aligned on representational alignmentcited by 143
  7. 2020Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistencycited by 78
  8. 2023Neither hype nor gloom do DNNs justicecited by 40
  9. 2022Do better ImageNet classifiers assess perceptual similarity better?cited by 34
  10. 2021AI4VIS: Survey on Artificial Intelligence Approaches for Data Visualizationcited by 32
  11. 2019ColorNet: Investigating the importance of color spaces for image classificationcited by 31
  12. 2020A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualizationcited by 28
  13. 2020Enriching ImageNet with Human Similarity Judgments and Psychological Embeddingscited by 23
  14. 2020Transforming Neural Network Visual Representations to Predict Human Judgments of Similaritycited by 16
  15. 2023A Systematic Performance Analysis of Deep Perceptual Loss Networks: Breaking Transfer Learning Conventionscited by 11
  16. 2023Are Deep Neural Networks Adequate Behavioural Models of Human Visual Perception?cited by 10
  17. 2022How Does Automation Shape the Process of Narrative Visualization: A Survey of Toolscited by 7
  18. 2019Discriminability Tests for Visualization Effectiveness and Scalabilitycited by 3
  19. 2021An Evaluation-Focused Framework for Visualization Recommendation Algorithmscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026VEIL: How Visual Encoding Hijacking Induces Bias In Vision Modelscited by 0