# No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling

arXiv: https://arxiv.org/abs/1804.09160 · HTML: https://mlsanity.com/paper/1804.09160/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2018.

## Built on by (most cited first, 30 of 46)

- [Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation](https://mlsanity.com/paper/1811.10092/) (2018)
- [Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning](https://mlsanity.com/paper/2108.03353/) (2021)
- ["Let Your Characters Tell Their Story": A Dataset for Character-Centric Narrative Understanding](https://mlsanity.com/paper/2109.05438/) (2021)
- [DataNarrative: Automated Data-Driven Storytelling with Visualizations and Texts](https://mlsanity.com/paper/2408.05346/) (2024)
- [Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning](https://mlsanity.com/paper/2108.05067/) (2021)
- [Commonsense Knowledge Aware Concept Selection For Diverse and Informative Visual Storytelling](https://mlsanity.com/paper/2102.02963/) (2021)
- [Multimodal Event Transformer for Image-guided Story Ending Generation](https://mlsanity.com/paper/2301.11357/) (2023)
- [Plot and Rework: Modeling Storylines for Visual Storytelling](https://mlsanity.com/paper/2105.06950/) (2021)
- [AdaNAT: Exploring Adaptive Policy for Token-Based Image Generation](https://mlsanity.com/paper/2409.00342/) (2024)
- [Scheduled Sampling Based on Decoding Steps for Neural Machine Translation](https://mlsanity.com/paper/2108.12963/) (2021)
- [VIST-GPT: Ushering in the Era of Visual Storytelling with LLMs?](https://mlsanity.com/paper/2504.19267/) (2025)
- [Confidence-Aware Scheduled Sampling for Neural Machine Translation](https://mlsanity.com/paper/2107.10427/) (2021)
- [L2C: Describing Visual Differences Needs Semantic Understanding of Individuals](https://mlsanity.com/paper/2102.01860/) (2021)
- [GROOViST: A Metric for Grounding Objects in Visual Storytelling](https://mlsanity.com/paper/2310.17770/) (2023)
- [RoViST:Learning Robust Metrics for Visual Storytelling](https://mlsanity.com/paper/2205.03774/) (2022)
- [Album Storytelling with Iterative Story-aware Captioning and Large Language Models](https://mlsanity.com/paper/2305.12943/) (2023)
- [Envisioning Narrative Intelligence: A Creative Visual Storytelling Anthology](https://mlsanity.com/paper/2310.04529/) (2023)
- [Ordered Attention for Coherent Visual Storytelling](https://mlsanity.com/paper/2108.02180/) (2021)
- [Every picture tells a story: Image-grounded controllable stylistic story generation](https://mlsanity.com/paper/2209.01638/) (2022)
- [Inverse Reinforcement Learning for Text Summarization](https://mlsanity.com/paper/2212.09917/) (2022)
- [Visual Storytelling with Question-Answer Plans](https://mlsanity.com/paper/2310.05295/) (2023)
- [SCO-VIST: Social Interaction Commonsense Knowledge-based Visual Storytelling](https://mlsanity.com/paper/2402.00319/) (2024)
- [Not (yet) the whole story: Evaluating Visual Storytelling Requires More than Measuring Coherence, Grounding, and Repetition](https://mlsanity.com/paper/2407.04559/) (2024)
- [DiMBERT: Learning Vision-Language Grounded Representations with Disentangled Multimodal-Attention](https://mlsanity.com/paper/2210.16431/) (2022)
- [Let Storytelling Tell Vivid Stories: An Expressive and Fluent Multimodal Storyteller](https://mlsanity.com/paper/2403.07301/) (2024)
- [ADSNet: Cross-Domain LTV Prediction with an Adaptive Siamese Network in Advertising](https://mlsanity.com/paper/2406.10517/) (2024)
- [Nested-Wasserstein Self-Imitation Learning for Sequence Generation](https://mlsanity.com/paper/2001.06944/) (2020)
- [An Adversarial Learning based Multi-Step Spoken Language Understanding System through Human-Computer Interaction](https://mlsanity.com/paper/2106.14611/) (2021)
- [A Study on Dense and Sparse (Visual) Rewards in Robot Policy Learning](https://mlsanity.com/paper/2108.03222/) (2021)
- [Knowledge-enriched Attention Network with Group-wise Semantic for Visual Storytelling](https://mlsanity.com/paper/2203.05346/) (2022)

## Abstract (arXiv)

Though impressive results have been achieved in visual captioning, the task of generating abstract stories from photo streams is still a little-tapped problem. Different from captions, stories have more expressive language styles and contain many imaginary concepts that do not appear in the images. Thus it poses challenges to behavioral cloning algorithms. Furthermore, due to the limitations of automatic metrics on evaluating story quality, reinforcement learning methods with hand-crafted rewards also face difficulties in gaining an overall performance boost. Therefore, we propose an Adversarial REward Learning (AREL) framework to learn an implicit reward function from human demonstrations, and then optimize policy search with the learned reward function. Though automatic eval- uation indicates slight performance boost over state-of-the-art (SOTA) methods in cloning expert behaviors, human evaluation shows that our approach achieves significant improvement in generating more human-like stories than SOTA systems.