# Image Memorability Prediction with Vision Transformers

arXiv: https://arxiv.org/abs/2301.08647 · HTML: https://mlsanity.com/paper/2301.08647/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 20 Jan 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [The Secrets of Salient Object Segmentation](https://mlsanity.com/paper/1406.2807/) (2014)
- [Are Convolutional Neural Networks or Transformers more like human vision?](https://mlsanity.com/paper/2105.07197/) (2021)
- [Defining Image Memorability using the Visual Memory Schema](https://mlsanity.com/paper/1903.02056/) (2019)
- [Embracing New Techniques in Deep Learning for Estimating Image Memorability](https://mlsanity.com/paper/2105.10598/) (2021)
- [Is Image Memorability Prediction Solved?](https://mlsanity.com/paper/1901.11420/) (2019)
- [Memorability: An image-computable measure of information utility](https://mlsanity.com/paper/2104.00805/) (2021)
- [The Resiliency of Memorability: A Predictor of Memory Separate from Attention and Priming](https://mlsanity.com/paper/1703.07738/) (2017)

## Built on by (most cited first, 9 of 9)

- [Long-Term Ad Memorability: Understanding & Generating Memorable Ads](https://mlsanity.com/paper/2309.00378/) (2023)
- [Teaching Human Behavior Improves Content Understanding Abilities Of LLMs](https://mlsanity.com/paper/2405.00942/) (2024)
- [Modeling Visual Memorability Assessment with Autoencoders Reveals Characteristics of Memorable Images](https://mlsanity.com/paper/2410.15235/) (2024)
- [Don't Judge Before You CLIP: A Unified Approach for Perceptual Tasks](https://mlsanity.com/paper/2503.13260/) (2025)
- [Unforgettable Lessons from Forgettable Images: Intra-Class Memorability Matters in Computer Vision](https://mlsanity.com/paper/2412.20761/) (2024)
- [MindMem: Multimodal for Predicting Advertisement Memorability Using LLMs and Deep Learning](https://mlsanity.com/paper/2502.18371/) (2025)
- [Correlates of Image Memorability in Vision Encoders: Activations, Attention Entropy, Patch Uniformity and Autoencoder Losses](https://mlsanity.com/paper/2509.01453/) (2025)
- [How to Take a Memorable Picture? Empowering Users with Actionable Feedback](https://mlsanity.com/paper/2602.21877/) (2026)
- [Human-Inspired Context-Selective Multimodal Memory for Social Robots](https://mlsanity.com/paper/2604.12081/) (2026)

## Abstract (arXiv)

Behavioral studies have shown that the memorability of images is similar across groups of people, suggesting that memorability is a function of the intrinsic properties of images, and is unrelated to people's individual experiences and traits. Deep learning networks can be trained on such properties and be used to predict memorability in new data sets. Convolutional neural networks (CNN) have pioneered image memorability prediction, but more recently developed vision transformer (ViT) models may have the potential to yield even better predictions. In this paper, we present the ViTMem, a new memorability model based on ViT, and evaluate memorability predictions obtained by it with state-of-the-art CNN-derived models. Results showed that ViTMem performed equal to or better than state-of-the-art models on all data sets. Additional semantic level analyses revealed that ViTMem is particularly sensitive to the semantic content that drives memorability in images. We conclude that ViTMem provides a new step forward, and propose that ViT-derived models can replace CNNs for computational prediction of image memorability. Researchers, educators, advertisers, visual designers and other interested parties can leverage the model to improve the memorability of their image material.