# The Influence of Audio on Video Memorability with an Audio Gestalt Regulated Video Memorability System

arXiv: https://arxiv.org/abs/2104.11568 · HTML: https://mlsanity.com/paper/2104.11568/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 44 references that are in the map.
- First posted on arXiv 23 Apr 2021.

## Builds on (most cited first, 10 of 10)

- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [Regularizing and Optimizing LSTM Language Models](https://mlsanity.com/paper/1708.02182/) (2017)
- [VideoMem: Constructing, Analyzing, Predicting Short-term and Long-term Video Memorability](https://mlsanity.com/paper/1812.01973/) (2018)
- [Multimodal Memorability: Modeling Effects of Semantics and Decay on Video Memorability](https://mlsanity.com/paper/2009.02568/) (2020)
- [Overview of MediaEval 2020 Predicting Media Memorability Task: What Makes a Video Memorable?](https://mlsanity.com/paper/2012.15650/) (2020)
- [Show and Recall: Learning What Makes Videos Memorable](https://mlsanity.com/paper/1707.05357/) (2017)
- [Leveraging Audio Gestalt to Predict Media Memorability](https://mlsanity.com/paper/2012.15635/) (2020)
- [HCU400: An Annotated Dataset for Exploring Aural Phenomenology Through Causal Uncertainty](https://mlsanity.com/paper/1811.06439/) (2018)
- [The Intrinsic Memorability of Everyday Sounds](https://mlsanity.com/paper/1811.07082/) (2018)

## Built on by (most cited first, 5 of 5)

- [Overview of The MediaEval 2022 Predicting Video Memorability Task](https://mlsanity.com/paper/2212.06516/) (2022)
- [Diffusing Surrogate Dreams of Video Scenes to Predict Video Memorability](https://mlsanity.com/paper/2212.09308/) (2022)
- [Predicting Media Memorability: Comparing Visual, Textual and Auditory Features](https://mlsanity.com/paper/2112.07969/) (2021)
- [Analysing the Memorability of a Procedural Crime-Drama TV Series, CSI](https://mlsanity.com/paper/2208.03479/) (2022)
- [Using Saliency and Cropping to Improve Video Memorability](https://mlsanity.com/paper/2309.11881/) (2023)

## Abstract (arXiv)

Memories are the tethering threads that tie us to the world, and memorability is the measure of their tensile strength. The threads of memory are spun from fibres of many modalities, obscuring the contribution of a single fibre to a thread's overall tensile strength. Unfurling these fibres is the key to understanding the nature of their interaction, and how we can ultimately create more meaningful media content. In this paper, we examine the influence of audio on video recognition memorability, finding evidence to suggest that it can facilitate overall video recognition memorability rich in high-level (gestalt) audio features. We introduce a novel multimodal deep learning-based late-fusion system that uses audio gestalt to estimate the influence of a given video's audio on its overall short-term recognition memorability, and selectively leverages audio features to make a prediction accordingly. We benchmark our audio gestalt based system on the Memento10k short-term video memorability dataset, achieving top-2 state-of-the-art results.