mlsanityData to 14 Oct 2026

What do different evaluation metrics tell us about saliency models?

arXiv 1604.03605 ↗First posted 12 Apr 2016cs.CVcited by 185

Not on the map. Why?

Abstract (arXiv)

How best to evaluate a saliency model's ability to predict where humans look in images is an open research question. The choice of evaluation metric depends on how saliency is defined and how the ground truth is represented. Metrics differ in how they rank saliency models, and this results from how false positives and false negatives are treated, whether viewing biases are accounted for, whether spatial deviations are factored in, and how the saliency maps are pre-processed. In this paper, we provide an analysis of 8 different evaluation metrics and their properties. With the help of systematic experiments and visualizations of metric computations, we add interpretability to saliency scores and more transparency to the evaluation of saliency models. Building off the differences in metric properties and behaviors, we make recommendations for metric selections under specific assumptions and for specific applications.

Key facts. Cited by 185 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Apr 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 2; 2020: 1; 2021: 29; 2022: 24; 2023: 34; 2024: 36; 2025: 32; 2026: 27.

185 papers.

Sort:

By area:

  1. 2023Rich Human Feedback for Text-to-Image Generationcited by 141
  2. 2022Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videoscited by 111
  3. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  4. 2021Guided Integrated Gradients: An Adaptive Path Method for Removing Noisecited by 102
  5. 2022Learning Affordance Grounding from Exocentric Imagescited by 82
  6. 2023Grounding 3D Object Affordance from 2D Interactions in Imagescited by 62
  7. 2023Which Tokens to Use? Investigating Token Reduction in Vision Transformerscited by 60
  8. 2021DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modelingcited by 56
  9. 2021TranSalNet: Towards perceptually relevant visual saliency predictioncited by 44
  10. 2022Deep Learning for Omnidirectional Vision: A Survey and New Perspectivescited by 38
  11. 2023Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attentioncited by 38
  12. 2024Alignment of Diffusion Models: Fundamentals, Challenges, and Futurecited by 35
38 more
  1. 2023Affordance Grounding from Demonstration Video to Target Imagecited by 34
  2. 2023Understanding 3D Object Interaction from a Single Imagecited by 33
  3. 2019MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learningcited by 29
  4. 2022A Review on Deep-Learning Algorithms for Fetal Ultrasound-Image Analysiscited by 29
  5. 2021Revisiting The Evaluation of Class Activation Mapping for Explainability: A Novel Metric and Experimental Analysiscited by 25
  6. 2021Learning Visual Affordance Grounding from Demonstration Videoscited by 25
  7. 2023TempSAL -- Uncovering Temporal Information for Deep Saliency Predictioncited by 25
  8. 2025UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulationcited by 23
  9. 2021Temporal-Spatial Feature Pyramid for Video Saliency Detectioncited by 22
  10. 2022Grounded Affordance from Exocentric Viewcited by 22
  11. 2023Unifying Top-down and Bottom-up Scanpath Prediction Using Transformerscited by 22
  12. 2021Human Attention in Fine-grained Classificationcited by 21
  13. 2022Target-absent Human Attentioncited by 20
  14. 2021Spatio-Temporal Self-Attention Network for Video Saliency Predictioncited by 19
  15. 2023Unsupervised Self-Driving Attention Prediction via Uncertainty Mining and Knowledge Embeddingcited by 19
  16. 2023Multi-label affordance mapping from egocentric visioncited by 18
  17. 2021Deep Saliency Prior for Reducing Visual Distractioncited by 17
  18. 2022Saliency in Augmented Realitycited by 17
  19. 2024A Survey of Imitation Learning Methods, Environments and Metricscited by 16
  20. 2022Panoramic Vision Transformer for Saliency Detection in 360deg Videoscited by 15
  21. 2023Fine-grained Affordance Annotation for Egocentric Hand-Object Interaction Videoscited by 15
  22. 2023CASP-Net: Rethinking Video Saliency Prediction from an Audio-VisualConsistency Perceptual Perspectivecited by 15
  23. 2024GazeXplain: Learning to Predict Natural Language Explanations of Visual Scanpathscited by 15
  24. 2022Rectify ViT Shortcut Learning by Visual Saliencycited by 14
  25. 2024Size-invariance Matters: Rethinking Metrics and Losses for Imbalanced Multi-object Salient Object Detectioncited by 14
  26. 2024WorldAfford: Affordance Grounding based on Natural Language Instructionscited by 14
  27. 2021Understanding Visual Saliency in Mobile User Interfacescited by 13
  28. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
  29. 2024DiffSal: Joint Audio and Video Learning for Diffusion Saliency Predictioncited by 13
  30. 2024HEIE: MLLM-Based Hierarchical Explainable AIGC Image Implausibility Evaluatorcited by 13
  31. 2024IG2: Integrated Gradient on Iterative Gradient Path for Feature Attributioncited by 12
  32. 2024SUM: Saliency Unification through Mamba for Visual Attention Modelingcited by 12
  33. 2024INTRA: Interaction Relationship-aware Weakly Supervised Affordance Groundingcited by 12
  34. 2021Bringing Generalization to Deep Multi-View Pedestrian Detectioncited by 11
  35. 2024PhyGrasp: Generalizing Robotic Grasping with Physics-informed Large Multimodal Modelscited by 11
  36. 2024Grounding 3D Scene Affordance From Egocentric Interactionscited by 11
  37. 2025ViSAGe: Video-to-Spatial Audio Generationcited by 11
  38. 2019Gaze Training by Modulated Dropout Improves Imitation Learningcited by 10

50 most cited of 185 All 185 (JSON)