# Hide-and-Tell: Learning to Bridge Photo Streams for Visual Storytelling

arXiv: https://arxiv.org/abs/2002.00774 · HTML: https://mlsanity.com/paper/2002.00774/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2020.

## Built on by (most cited first, 13 of 13)

- [Commonsense Knowledge Aware Concept Selection For Diverse and Informative Visual Storytelling](https://mlsanity.com/paper/2102.02963/) (2021)
- [Multimodal Event Transformer for Image-guided Story Ending Generation](https://mlsanity.com/paper/2301.11357/) (2023)
- [Plot and Rework: Modeling Storylines for Visual Storytelling](https://mlsanity.com/paper/2105.06950/) (2021)
- [Learning Dynamic Style Kernels for Artistic Style Transfer](https://mlsanity.com/paper/2304.00414/) (2023)
- [SCO-VIST: Social Interaction Commonsense Knowledge-based Visual Storytelling](https://mlsanity.com/paper/2402.00319/) (2024)
- [Disentangled Representation Learning for Controllable Person Image Generation](https://mlsanity.com/paper/2312.05798/) (2023)
- [Knowledge-enriched Attention Network with Group-wise Semantic for Visual Storytelling](https://mlsanity.com/paper/2203.05346/) (2022)
- [Text-Only Training for Visual Storytelling](https://mlsanity.com/paper/2308.08881/) (2023)
- [Story Generation from Visual Inputs: Techniques, Related Tasks, and Challenges](https://mlsanity.com/paper/2406.02748/) (2024)
- [A look under the hood of the Interactive Deep Learning Enterprise (No-IDLE)](https://mlsanity.com/paper/2406.19054/) (2024)
- [DiffuVST: Narrating Fictional Scenes with Global-History-Guided Denoising Models](https://mlsanity.com/paper/2312.07066/) (2023)
- [TARN-VIST: Topic Aware Reinforcement Network for Visual Storytelling](https://mlsanity.com/paper/2403.11550/) (2024)
- [From Image Captioning to Visual Storytelling](https://mlsanity.com/paper/2508.14045/) (2025)

## Abstract (arXiv)

Visual storytelling is a task of creating a short story based on photo streams. Unlike existing visual captioning, storytelling aims to contain not only factual descriptions, but also human-like narration and semantics. However, the VIST dataset consists only of a small, fixed number of photos per story. Therefore, the main challenge of visual storytelling is to fill in the visual gap between photos with narrative and imaginative story. In this paper, we propose to explicitly learn to imagine a storyline that bridges the visual gap. During training, one or more photos is randomly omitted from the input stack, and we train the network to produce a full plausible story even with missing photo(s). Furthermore, we propose for visual storytelling a hide-and-tell model, which is designed to learn non-local relations across the photo streams and to refine and improve conventional RNN-based models. In experiments, we show that our scheme of hide-and-tell, and the network design are indeed effective at storytelling, and that our model outperforms previous state-of-the-art methods in automatic metrics. Finally, we qualitatively show the learned ability to interpolate storyline over visual gaps.