# Embracing New Techniques in Deep Learning for Estimating Image Memorability

arXiv: https://arxiv.org/abs/2105.10598 · HTML: https://mlsanity.com/paper/2105.10598/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 56 references that are in the map.
- First posted on arXiv 21 May 2021.

## Builds on (most cited first, 3 of 3)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [AMNet: Memorability Estimation with Attention](https://mlsanity.com/paper/1804.03115/) (2018)

## Built on by (most cited first, 10 of 10)

- [Image Memorability Prediction with Vision Transformers](https://mlsanity.com/paper/2301.08647/) (2023)
- [Controlling Memorability of Face Images](https://mlsanity.com/paper/2202.11896/) (2022)
- [Modeling Visual Memorability Assessment with Autoencoders Reveals Characteristics of Memorable Images](https://mlsanity.com/paper/2410.15235/) (2024)
- [Don't Judge Before You CLIP: A Unified Approach for Perceptual Tasks](https://mlsanity.com/paper/2503.13260/) (2025)
- [Unforgettable Lessons from Forgettable Images: Intra-Class Memorability Matters in Computer Vision](https://mlsanity.com/paper/2412.20761/) (2024)
- [Complexity in Complexity: Understanding Visual Complexity Through Structure, Color, and Surprise](https://mlsanity.com/paper/2501.15890/) (2025)
- [Visual Interestingness Decoded: How GPT-4o Mirrors Human Interests](https://mlsanity.com/paper/2510.13316/) (2025)
- [Correlates of Image Memorability in Vision Encoders: Activations, Attention Entropy, Patch Uniformity and Autoencoder Losses](https://mlsanity.com/paper/2509.01453/) (2025)
- [The Persistence of Cultural Memory: Investigating Multimodal Iconicity in Diffusion Models](https://mlsanity.com/paper/2511.11435/) (2025)
- [Human-Inspired Context-Selective Multimodal Memory for Social Robots](https://mlsanity.com/paper/2604.12081/) (2026)

## Abstract (arXiv)

Various work has suggested that the memorability of an image is consistent across people, and thus can be treated as an intrinsic property of an image. Using computer vision models, we can make specific predictions about what people will remember or forget. While older work has used now-outdated deep learning architectures to predict image memorability, innovations in the field have given us new techniques to apply to this problem. Here, we propose and evaluate five alternative deep learning models which exploit developments in the field from the last five years, largely the introduction of residual neural networks, which are intended to allow the model to use semantic information in the memorability estimation process. These new models were tested against the prior state of the art with a combined dataset built to optimize both within-category and across-category predictions. Our findings suggest that the key prior memorability network had overstated its generalizability and was overfit on its training set. Our new models outperform this prior model, leading us to conclude that Residual Networks outperform simpler convolutional neural networks in memorability regression. We make our new state-of-the-art model readily available to the research community, allowing memory researchers to make predictions about memorability on a wider range of images.