# Predicting Visual Importance Across Graphic Design Types

arXiv: https://arxiv.org/abs/2008.02912 · HTML: https://mlsanity.com/paper/2008.02912/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Aug 2020.

## Built on by (most cited first, 30 of 30)

- [Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning](https://mlsanity.com/paper/2108.03353/) (2021)
- [TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction](https://mlsanity.com/paper/2301.02315/) (2023)
- [UICrit: Enhancing Automated Design Evaluation with a UICritique Dataset](https://mlsanity.com/paper/2407.08850/) (2024)
- [GazeXplain: Learning to Predict Natural Language Explanations of Visual Scanpaths](https://mlsanity.com/paper/2408.02788/) (2024)
- [Beyond Average: Individualized Visual Scanpath Prediction](https://mlsanity.com/paper/2404.12235/) (2024)
- [SUM: Saliency Unification through Mamba for Visual Attention Modeling](https://mlsanity.com/paper/2406.17815/) (2024)
- [Towards Better User Studies in Computer Graphics and Vision](https://mlsanity.com/paper/2206.11461/) (2022)
- [Predicting and Explaining Mobile UI Tappability with Vision Modeling and Saliency Analysis](https://mlsanity.com/paper/2204.02448/) (2022)
- [Trends, Applications, and Challenges in Human Attention Modelling](https://mlsanity.com/paper/2402.18673/) (2024)
- [EyeFormer: Predicting Personalized Scanpaths with Transformer-Guided Reinforcement Learning](https://mlsanity.com/paper/2404.10163/) (2024)
- [UniAR: A Unified model for predicting human Attention and Responses on visual content](https://mlsanity.com/paper/2312.10175/) (2023)
- [DesignProbe: A Graphic Design Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2404.14801/) (2024)
- [Scanpath Prediction on Information Visualisations](https://mlsanity.com/paper/2112.02340/) (2021)
- [What Do Deep Saliency Models Learn about Visual Attention?](https://mlsanity.com/paper/2310.09679/) (2023)
- [Infographics Wizard: Flexible Infographics Authoring and Design Exploration](https://mlsanity.com/paper/2204.09904/) (2022)
- [Predicting Visual Attention in Graphic Design Documents](https://mlsanity.com/paper/2407.02439/) (2024)
- [User-Centric Semi-Automated Infographics Authoring and Recommendation](https://mlsanity.com/paper/2108.11914/) (2021)
- [GIRT-Model: Automated Generation of Issue Report Templates](https://mlsanity.com/paper/2402.02632/) (2024)
- [Impact of Design Decisions in Scanpath Modeling](https://mlsanity.com/paper/2405.08981/) (2024)
- [Synthetic Heuristic Evaluation: A Comparison between AI- and Human-Powered Usability Evaluation](https://mlsanity.com/paper/2507.02306/) (2025)
- [Unlocking Comics: The AI4VA Dataset for Visual Understanding](https://mlsanity.com/paper/2410.20459/) (2024)
- [Tell Me Without Telling Me: Two-Way Prediction of Visualization Literacy and Visual Attention](https://mlsanity.com/paper/2508.03713/) (2025)
- [Attend to Anything: Foundation Model for Unified Human Attention Modeling](https://mlsanity.com/paper/2606.03540/) (2026)
- [Learning Aesthetic Layouts via Visual Guidance](https://mlsanity.com/paper/2107.06262/) (2021)
- [Few-shot Personalized Saliency Prediction Based on Interpersonal Gaze Patterns](https://mlsanity.com/paper/2307.02799/) (2023)
- [Questionnaires for Everyone: Streamlining Cross-Cultural Questionnaire Adaptation with GPT-Based Translation Quality Evaluation](https://mlsanity.com/paper/2407.20608/) (2024)
- [A Comparative Study of Scanpath Models in Graph-Based Visualization](https://mlsanity.com/paper/2503.24160/) (2025)
- [GRAPHIC--Guidelines for Reviewing Algorithmic Practices in Human-centred Design and Interaction for Creativity](https://mlsanity.com/paper/2511.17443/) (2025)
- [TASTE: A Designer-Annotated Multi-Dimensional Preference Dataset for AI-Generated Graphic Design](https://mlsanity.com/paper/2605.20731/) (2026)
- [OpenVAM: Open-World Visual Attention Modeling with VLMs](https://mlsanity.com/paper/2609.31364/) (2026)

## Abstract (arXiv)

This paper introduces a Unified Model of Saliency and Importance (UMSI), which learns to predict visual importance in input graphic designs, and saliency in natural images, along with a new dataset and applications. Previous methods for predicting saliency or visual importance are trained individually on specialized datasets, making them limited in application and leading to poor generalization on novel image classes, while requiring a user to know which model to apply to which input. UMSI is a deep learning-based model simultaneously trained on images from different design classes, including posters, infographics, mobile UIs, as well as natural images, and includes an automatic classification module to classify the input. This allows the model to work more effectively without requiring a user to label the input. We also introduce Imp1k, a new dataset of designs annotated with importance information. We demonstrate two new design interfaces that use importance prediction, including a tool for adjusting the relative importance of design elements, and a tool for reflowing designs to new aspect ratios while preserving visual importance. The model, code, and importance dataset are available at https://predimportance.mit.edu .