Belief Revision based Caption Re-ranker with Visual Semantic Information
Not on the map. Why?
Abstract (arXiv)
In this work, we focus on improving the captions generated by image-caption generation systems. We propose a novel re-ranking approach that leverages visual-semantic measures to identify the ideal caption that maximally captures the visual information in the image. Our re-ranker utilizes the Belief Revision framework (Blok et al., 2003) to calibrate the original likelihood of the top-n captions by explicitly exploiting the semantic relatedness between the depicted caption and the visual context. Our experiments demonstrate the utility of our approach, where we observe that our re-ranker can enhance the performance of a typical image-captioning system without the necessity of any additional training or fine-tuning.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 44 of its 61 references that are in the map. First posted on arXiv 16 Sep 2022.
Builds on
44 of 61 references.
References · 44
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
32 more
- 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
- 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
- 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
- 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 968
- 2014Show and Tell: A Neural Image Caption Generatorcited by 879
- 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
- 2016Self-critical Sequence Training for Image Captioningcited by 547
- 2019Meshed-Memory Transformer for Image Captioningcited by 343
- 2019Attention on Attention for Image Captioningcited by 289
- 201912-in-1: Multi-Task Vision and Language Representation Learningcited by 278
- 2016Image Captioning with Semantic Attentioncited by 260
- 2016Speed/accuracy trade-offs for modern convolutional object detectorscited by 198
- 2019Image Captioning: Transforming Objects into Wordscited by 171
- 2021To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translationcited by 163
- 2020Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metricscited by 158
- 2014From Captions to Visual Concepts and Backcited by 109
- 2017SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publicationscited by 102
- 2017Deep Keyphrase Generationcited by 102
- 2020Contrastive Learning for Weakly Supervised Phrase Groundingcited by 102
- 2017Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Imagescited by 101
- 2020Say As You Wish: Fine-grained Control of Image Caption Generation with Abstract Scene Graphscited by 87
- 2019Comparison of Diverse Decoding Methods from Conditional Language Modelscited by 78
- 2018Show, Control and Tell: A Framework for Generating Controllable and Grounded Captionscited by 65
- 2017Towards Diverse and Natural Image Descriptions via a Conditional GANcited by 59
- 2018Fast, Diverse and Accurate Image Captioning Guided By Part-of-Speechcited by 55
- 2019Describing like humans: on diversity in image captioningcited by 46
- 2017Context-aware Captions from Context-agnostic Supervisioncited by 43
- 2017Deep Reinforcement Learning-based Image Captioning with Embedding Rewardcited by 40
- 2021Consensus Graph Representation Learning for Better Grounded Image Captioningcited by 28
- 2020Compare and Reweight: Distinctive Image Captioning Using Similar Images Setscited by 27
- 2018Object Counts! Bringing Explicit Detections Back into Image Captioningcited by 5
- 2018Visual Re-ranking with Natural Language Understanding for Text Spottingcited by 4
Built on by
Citing papers per year: 2023: 2.
2 papers.