# Multimodal Memorability: Modeling Effects of Semantics and Decay on Video Memorability

arXiv: https://arxiv.org/abs/2009.02568 · HTML: https://mlsanity.com/paper/2009.02568/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Sep 2020.

## Built on by (most cited first, 23 of 23)

- [Long-Term Ad Memorability: Understanding & Generating Memorable Ads](https://mlsanity.com/paper/2309.00378/) (2023)
- [Memorability: An image-computable measure of information utility](https://mlsanity.com/paper/2104.00805/) (2021)
- [The Influence of Audio on Video Memorability with an Audio Gestalt Regulated Video Memorability System](https://mlsanity.com/paper/2104.11568/) (2021)
- [Overview of The MediaEval 2021 Predicting Media Memorability Task](https://mlsanity.com/paper/2112.05982/) (2021)
- [Overview of The MediaEval 2022 Predicting Video Memorability Task](https://mlsanity.com/paper/2212.06516/) (2022)
- [Diffusing Surrogate Dreams of Video Scenes to Predict Video Memorability](https://mlsanity.com/paper/2212.09308/) (2022)
- [Hypergraph Multi-modal Large Language Model: Exploiting EEG and Eye-tracking Modalities to Evaluate Heterogeneous Responses for Video Understanding](https://mlsanity.com/paper/2407.08150/) (2024)
- [Predicting Media Memorability: Comparing Visual, Textual and Auditory Features](https://mlsanity.com/paper/2112.07969/) (2021)
- [Experiences from the MediaEval Predicting Media Memorability Task](https://mlsanity.com/paper/2212.03955/) (2022)
- [Teaching Human Behavior Improves Content Understanding Abilities Of LLMs](https://mlsanity.com/paper/2405.00942/) (2024)
- [Analysing the Memorability of a Procedural Crime-Drama TV Series, CSI](https://mlsanity.com/paper/2208.03479/) (2022)
- [Seeing Eye to AI: Comparing Human Gaze and Model Attention in Video Memorability](https://mlsanity.com/paper/2311.16484/) (2023)
- [Introducing MeMo: A Multimodal Dataset for Memory Modelling in Multiparty Conversations](https://mlsanity.com/paper/2409.13715/) (2024)
- [Unforgettable Lessons from Forgettable Images: Intra-Class Memorability Matters in Computer Vision](https://mlsanity.com/paper/2412.20761/) (2024)
- [Shared memories driven by the intrinsic memorability of items](https://mlsanity.com/paper/2104.06937/) (2021)
- [Using Saliency and Cropping to Improve Video Memorability](https://mlsanity.com/paper/2309.11881/) (2023)
- [Memories in the Making: Predicting Video Memorability with Encoding Phase EEG](https://mlsanity.com/paper/2309.16704/) (2023)
- [MindMem: Multimodal for Predicting Advertisement Memorability Using LLMs and Deep Learning](https://mlsanity.com/paper/2502.18371/) (2025)
- [Enhancing Video Memorability Prediction with Text-Motion Cross-modal Contrastive Loss and Its Application in Video Summarization](https://mlsanity.com/paper/2506.08649/) (2025)
- [The Emotion-Memory Link: Do Memorability Annotations Matter for Intelligent Systems?](https://mlsanity.com/paper/2507.14084/) (2025)
- [Unsupervised Memorability Modeling from Tip-of-the-Tongue Retrieval Queries](https://mlsanity.com/paper/2511.20854/) (2025)
- [How to Take a Memorable Picture? Empowering Users with Actionable Feedback](https://mlsanity.com/paper/2602.21877/) (2026)
- [Predicted Cortex Is Not a Domain-General Prior: A Matched-Control Audit of Brain-Encoding Features for Video Memorability](https://mlsanity.com/paper/2607.16292/) (2026)

## Abstract (arXiv)

A key capability of an intelligent system is deciding when events from past experience must be remembered and when they can be forgotten. Towards this goal, we develop a predictive model of human visual event memory and how those memories decay over time. We introduce Memento10k, a new, dynamic video memorability dataset containing human annotations at different viewing delays. Based on our findings we propose a new mathematical formulation of memorability decay, resulting in a model that is able to produce the first quantitative estimation of how a video decays in memory over time. In contrast with previous work, our model can predict the probability that a video will be remembered at an arbitrary delay. Importantly, our approach combines visual and semantic information (in the form of textual captions) to fully represent the meaning of events. Our experiments on two video memorability benchmarks, including Memento10k, show that our model significantly improves upon the best prior approach (by 12% on average).