mlsanityData to 14 Oct 2026

UniAR: A Unified model for predicting human Attention and Responses on visual content

arXiv 2312.10175 ↗First posted 15 Dec 2023cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Progress in human behavior modeling involves understanding both implicit, early-stage perceptual behavior, such as human attention, and explicit, later-stage behavior, such as subjective preferences or likes. Yet most prior research has focused on modeling implicit and explicit human behavior in isolation; and often limited to a specific type of visual content. We propose UniAR -- a unified model of human attention and preference behavior across diverse visual content. UniAR leverages a multimodal transformer to predict subjective feedback, such as satisfaction or aesthetic quality, along with the underlying human attention or interaction heatmaps and viewing order. We train UniAR on diverse public datasets spanning natural images, webpages, and graphic designs, and achieve SOTA performance on multiple benchmarks across various image domains and behavior modeling tasks. Potential applications include providing instant feedback on the effectiveness of UIs/visual content, and enabling designers and content-creation models to optimize their creation for human-centric improvements.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 37 of its 84 references that are in the map. First posted on arXiv 15 Dec 2023.

Builds on

37 of 84 references.

References · 37

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  4. 2023PaLM 2 Technical Reportcited by 1,162
  5. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  6. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  7. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
  8. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  9. 2019KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessmentcited by 355
  10. 2019Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Networkcited by 302
  11. 2016Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessmentcited by 225
  12. 2016What do different evaluation metrics tell us about saliency models?cited by 185
25 more
  1. 2022A Unified Sequence Interface for Vision Taskscited by 152
  2. 2020MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessmentcited by 144
  3. 2016Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Modelcited by 91
  4. 2020Unified Image and Video Saliency Modelingcited by 75
  5. 2017Deep Visual Attention Predictioncited by 70
  6. 2017SalGAN: Visual Saliency Prediction with Generative Adversarial Networkscited by 65
  7. 2022Spotlight: Mobile UI Understanding using Vision-Language Models with a Focuscited by 62
  8. 2014Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNetcited by 60
  9. 2021DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modelingcited by 56
  10. 2016DeepGaze II: Reading fixations from deep features trained on object recognitioncited by 55
  11. 2020Predicting Goal-directed Human Attention Using Inverse Reinforcement Learningcited by 53
  12. 2015CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Researchcited by 52
  13. 2018EML-NET:An Expandable Multi-Layer NETwork for Saliency Predictioncited by 51
  14. 2023Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attentioncited by 38
  15. 2020Predicting Visual Importance Across Graphic Design Typescited by 30
  16. 2017A Probabilistic Quality Representation Approach to Deep Blind Image Quality Predictioncited by 28
  17. 2017Learning Visual Importance for Graphic Designs and Data Visualizationscited by 22
  18. 2019A Dilated Inception Network for Visual Saliency Predictioncited by 22
  19. 2020Tidying Deep Saliency Prediction Architecturescited by 20
  20. 2022Target-absent Human Attentioncited by 20
  21. 2021Deep Saliency Prior for Reducing Visual Distractioncited by 17
  22. 2019Understanding and Visualizing Deep Visual Saliency Modelscited by 13
  23. 2021Understanding Visual Saliency in Mobile User Interfacescited by 13
  24. 2021SalFBNet: Learning Pseudo-Saliency Distribution via Feedback Convolutional Networkscited by 9
  25. 2022Predicting and Explaining Mobile UI Tappability with Vision Modeling and Saliency Analysiscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025Modeling Human Gaze Behavior with Diffusion Models for Unified Scanpath Predictioncited by 7
  2. 2025Modeling Saliency Dataset Biascited by 4
  3. 2025CrossGuard: Safeguarding MLLMs against Joint-Modal Implicit Malicious Attackscited by 4
  4. 2025OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responsescited by 3
  5. 2025Integrating Cognitive Processing Signals into Language Models: A Review of Advances, Applications and Future Directionscited by 2
  6. 2026Attend to Anything: Foundation Model for Unified Human Attention Modelingcited by 1
  7. 2026OpenVAM: Open-World Visual Attention Modeling with VLMscited by 0