# Diffusing Surrogate Dreams of Video Scenes to Predict Video Memorability

arXiv: https://arxiv.org/abs/2212.09308 · HTML: https://mlsanity.com/paper/2212.09308/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 17 references that are in the map.
- First posted on arXiv 19 Dec 2022.

## Builds on (most cited first, 7 of 7)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Multimodal Memorability: Modeling Effects of Semantics and Decay on Video Memorability](https://mlsanity.com/paper/2009.02568/) (2020)
- [Leveraging Audio Gestalt to Predict Media Memorability](https://mlsanity.com/paper/2012.15635/) (2020)
- [The Influence of Audio on Video Memorability with an Audio Gestalt Regulated Video Memorability System](https://mlsanity.com/paper/2104.11568/) (2021)
- [Overview of The MediaEval 2022 Predicting Video Memorability Task](https://mlsanity.com/paper/2212.06516/) (2022)

## Built on by (most cited first, 4 of 4)

- [Diffusion Action Segmentation](https://mlsanity.com/paper/2303.17959/) (2023)
- [Using Saliency and Cropping to Improve Video Memorability](https://mlsanity.com/paper/2309.11881/) (2023)
- [Enhancing Video Memorability Prediction with Text-Motion Cross-modal Contrastive Loss and Its Application in Video Summarization](https://mlsanity.com/paper/2506.08649/) (2025)
- [CPKD: Clinical Prior Knowledge-Constrained Diffusion Models for Surgical Phase Recognition in Endoscopic Submucosal Dissection](https://mlsanity.com/paper/2507.03295/) (2025)

## Abstract (arXiv)

As part of the MediaEval 2022 Predicting Video Memorability task we explore the relationship between visual memorability, the visual representation that characterises it, and the underlying concept portrayed by that visual representation. We achieve state-of-the-art memorability prediction performance with a model trained and tested exclusively on surrogate dream images, elevating concepts to the status of a cornerstone memorability feature, and finding strong evidence to suggest that the intrinsic memorability of visual content can be distilled to its underlying concept or meaning irrespective of its specific visual representational.