# Unforgettable Lessons from Forgettable Images: Intra-Class Memorability Matters in Computer Vision

arXiv: https://arxiv.org/abs/2412.20761 · HTML: https://mlsanity.com/paper/2412.20761/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 28 of its 60 references that are in the map.
- First posted on arXiv 30 Dec 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [GANalyze: Toward Visual Definitions of Cognitive Image Properties](https://mlsanity.com/paper/1906.10112/) (2019)
- [Don't forget, there is more than forgetting: new metrics for Continual Learning](https://mlsanity.com/paper/1810.13166/) (2018)
- [Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfer](https://mlsanity.com/paper/2211.00789/) (2022)
- [CCIL: Continuity-based Data Augmentation for Corrective Imitation Learning](https://mlsanity.com/paper/2310.12972/) (2023)
- [An Investigation of Replay-based Approaches for Continual Learning](https://mlsanity.com/paper/2108.06758/) (2021)
- [VideoMem: Constructing, Analyzing, Predicting Short-term and Long-term Video Memorability](https://mlsanity.com/paper/1812.01973/) (2018)
- [Multimodal Memorability: Modeling Effects of Semantics and Decay on Video Memorability](https://mlsanity.com/paper/2009.02568/) (2020)
- [Defining Image Memorability using the Visual Memory Schema](https://mlsanity.com/paper/1903.02056/) (2019)
- [Embracing New Techniques in Deep Learning for Estimating Image Memorability](https://mlsanity.com/paper/2105.10598/) (2021)
- [Is Image Memorability Prediction Solved?](https://mlsanity.com/paper/1901.11420/) (2019)
- [Learning to Learn: How to Continuously Teach Humans and Machines](https://mlsanity.com/paper/2211.15470/) (2022)
- [Image Memorability Prediction with Vision Transformers](https://mlsanity.com/paper/2301.08647/) (2023)
- [Memorability: An image-computable measure of information utility](https://mlsanity.com/paper/2104.00805/) (2021)
- [Synthetic Human Memories: AI-Edited Images and Videos Can Implant False Memories and Distort Recollection](https://mlsanity.com/paper/2409.08895/) (2024)
- [Understanding and Predicting the Memorability of Outdoor Natural Scenes](https://mlsanity.com/paper/1810.06679/) (2018)
- [Unveiling the Tapestry: the Interplay of Generalization and Forgetting in Continual Learning](https://mlsanity.com/paper/2211.11174/) (2022)
- [Integrating Curricula with Replays: Its Effects on Continual Learning](https://mlsanity.com/paper/2307.05747/) (2023)
- [Decoding the Enigma: Benchmarking Humans and AIs on the Many Facets of Working Memory](https://mlsanity.com/paper/2307.10768/) (2023)
- [Feature representations useful for predicting image memorability](https://mlsanity.com/paper/2303.07679/) (2023)

## Built on by (most cited first, 1 of 1)

- [From Pixels to Feelings: Aligning MLLMs with Human Cognitive Perception of Images](https://mlsanity.com/paper/2511.22805/) (2025)

## Abstract (arXiv)

We introduce intra-class memorability, where certain images within the same class are more memorable than others despite shared category characteristics. To investigate what features make one object instance more memorable than others, we design and conduct human behavior experiments, where participants are shown a series of images, and they must identify when the current image matches the image presented a few steps back in the sequence. To quantify memorability, we propose the Intra-Class Memorability score (ICMscore), a novel metric that incorporates the temporal intervals between repeated image presentations into its calculation. Furthermore, we curate the Intra-Class Memorability Dataset (ICMD), comprising over 5,000 images across ten object classes with their ICMscores derived from 2,000 participants' responses. Subsequently, we demonstrate the usefulness of ICMD by training AI models on this dataset for various downstream tasks: memorability prediction, image recognition, continual learning, and memorability-controlled image editing. Surprisingly, high-ICMscore images impair AI performance in image recognition and continual learning tasks, while low-ICMscore images improve outcomes in these tasks. Additionally, we fine-tune a state-of-the-art image diffusion model on ICMD image pairs with and without masked semantic objects. The diffusion model can successfully manipulate image elements to enhance or reduce memorability. Our contributions open new pathways in understanding intra-class memorability by scrutinizing fine-grained visual features behind the most and least memorable images and laying the groundwork for real-world applications in computer vision. We will release all code, data, and models publicly.