# Unsupervised Memorability Modeling from Tip-of-the-Tongue Retrieval Queries

arXiv: https://arxiv.org/abs/2511.20854 · HTML: https://mlsanity.com/paper/2511.20854/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 56 references that are in the map.
- First posted on arXiv 25 Nov 2025.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks](https://mlsanity.com/paper/2410.05160/) (2024)
- [InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling](https://mlsanity.com/paper/2501.12386/) (2025)
- [GME: Improving Universal Multimodal Retrieval by Multimodal LLMs](https://mlsanity.com/paper/2412.16855/) (2024)
- [PaddleOCR 3.0 Technical Report](https://mlsanity.com/paper/2507.05595/) (2025)
- [Understanding the Repeat Curse in Large Language Models from a Feature Perspective](https://mlsanity.com/paper/2504.14218/) (2025)
- [VideoMem: Constructing, Analyzing, Predicting Short-term and Long-term Video Memorability](https://mlsanity.com/paper/1812.01973/) (2018)
- [Multimodal Memorability: Modeling Effects of Semantics and Decay on Video Memorability](https://mlsanity.com/paper/2009.02568/) (2020)
- [Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification](https://mlsanity.com/paper/2101.07124/) (2021)
- [An Annotated Video Dataset for Computing Video Memorability](https://mlsanity.com/paper/2112.02303/) (2021)
- [Long-Term Ad Memorability: Understanding & Generating Memorable Ads](https://mlsanity.com/paper/2309.00378/) (2023)
- [When the Music Stops: Tip-of-the-Tongue Retrieval for Music](https://mlsanity.com/paper/2305.14072/) (2023)
- [HICEM: A High-Coverage Emotion Model for Artificial Emotional Intelligence](https://mlsanity.com/paper/2206.07593/) (2022)

## Abstract (arXiv)

Visual content memorability has intrigued the scientific community for decades, with applications ranging widely, from understanding nuanced aspects of human memory to enhancing content design. A significant challenge in progressing the field lies in the expensive process of collecting memorability annotations from humans. This limits the diversity and scalability of datasets for modeling visual content memorability. Most existing datasets are limited to collecting aggregate memorability scores for visual content, not capturing the nuanced memorability signals present in natural, open-ended recall descriptions. In this work, we introduce the first large-scale unsupervised dataset designed explicitly for modeling visual memorability signals, containing over 82,000 videos, accompanied by descriptive recall data. We leverage tip-of-the-tongue (ToT) retrieval queries from online platforms such as Reddit. We demonstrate that our unsupervised dataset provides rich signals for two memorability-related tasks: recall generation and ToT retrieval. Large vision-language models fine-tuned on our dataset outperform state-of-the-art models such as GPT-4o in generating open-ended memorability descriptions for visual content. We also employ a contrastive training strategy to create the first model capable of performing multimodal ToT retrieval. Our dataset and models present a novel direction, facilitating progress in visual content memorability research.