mlsanityData to 7 Oct 2026

Reinforced Video Captioning with Entailment Rewards

arXiv 1708.02300 ↗First posted 7 Aug 2017cs.CL cs.AI cs.CVcited by 21

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

Sequence-to-sequence models have shown promising improvements on the temporal task of video captioning, but they optimize word-level cross-entropy loss during training. First, using policy gradient and mixed-loss methods for reinforcement learning, we directly optimize sentence-level task-based metrics (as rewards), achieving significant improvements over the baseline, based on both automatic metrics and human evaluation on multiple datasets. Next, we propose a novel entailment-enhanced reward (CIDEnt) that corrects phrase-matching based metrics (such as CIDEr) to only allow for logically-implied partial matches and avoid contradictions, achieving further significant improvements over the CIDEr-reward model. Overall, our CIDEnt-reward model achieves the new state-of-the-art on the MSR-VTT dataset.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 26 of its 40 references that are in the map. First posted on arXiv 7 Aug 2017.

Builds on

26 of 40 references.

References · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  4. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  5. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  6. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  7. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  8. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  9. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  10. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  11. 2016Self-critical Sequence Training for Image Captioningcited by 544
  12. 2017A Deep Reinforced Model for Abstractive Summarizationcited by 364
14 more
  1. 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
  2. 2016A Decomposable Attention Model for Natural Language Inferencecited by 214
  3. 2014Recurrent Neural Network Regularizationcited by 189
  4. 2016Long Short-Term Memory-Networks for Machine Readingcited by 161
  5. 2015Sequence to Sequence -- Video to Textcited by 157
  6. 2016Improved Image Captioning via Policy Gradient optimization of SPIDErcited by 112
  7. 2014Translating Videos to Natural Language Using Deep Recurrent Neural Networkscited by 95
  8. 2015Describing Videos by Exploiting Temporal Structurecited by 88
  9. 2015Reasoning about Entailment with Neural Attentioncited by 68
  10. 2015Jointly Modeling Embedding and Translation to Bridge Video and Languagecited by 53
  11. 2015Hierarchical Recurrent Neural Encoder for Video Representation with Application to Captioningcited by 30
  12. 2017Multi-Task Video Captioning with Video and Entailment Generationcited by 17
  13. 2015Reinforcement Learning Neural Turing Machines - Revisedcited by 11
  14. 2016Improving LSTM-based Video Description with Linguistic Knowledge Mined from Textcited by 11

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 10; 2022: 4; 2023: 2; 2024: 1; 2025: 3.

21 papers.

Sort:

By area:

  1. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 130
  2. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 118
  3. 2022On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgettingcited by 80
  4. 2023A Review of Deep Learning for Video Captioningcited by 28
  5. 2021Relational Graph Learning for Grounded Video Description Generationcited by 12
  6. 2021ENTRUST: Argument Reframing with Language Models and Entailmentcited by 10
  7. 2021Energy-Based Models for Code Generation under Compilability Constraintscited by 10
  8. 2021Controllable Video Captioning with an Exemplar Sentencecited by 9
  9. 2021The MSR-Video to Text Dataset with Clean Annotationscited by 8
  10. 2025IntentVCNet: Bridging Spatio-Temporal Gaps for Intention-Oriented Controllable Video Captioningcited by 7
  11. 2021A Comprehensive Review of the Video-to-Text Problemcited by 5
  12. 2021Automatic Document Sketching: Generating Drafts from Analogous Textscited by 4
9 more
  1. 2020Nested-Wasserstein Self-Imitation Learning for Sequence Generationcited by 3
  2. 2022Variational Stacked Local Attention Networks for Diverse Video Captioningcited by 3
  3. 2025DANTE-AD: Dual-Vision Attention Network for Long-Term Audio Descriptioncited by 3
  4. 2021Syntax Customized Video Captioning by Imitating Exemplar Sentencescited by 2
  5. 2025Reward Models are Metrics in a Trench Coatcited by 2
  6. 2021The Role of the Input in Natural Language Video Descriptioncited by 1
  7. 2021Visual-aware Attention Dual-stream Decoder for Video Captioningcited by 1
  8. 2022METEOR Guided Divergence for Video Captioningcited by 1
  9. 2024Diverse Sign Language Translationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioning
  2. 2025CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning
  3. 2026Claim-Level Rubric Rewards for Video Caption Reinforcement Learning
  4. 2019Transfer Reward Learning for Policy Gradient-Based Text Generation
  5. 2024Fluent and Accurate Image Captioning with a Self-Trained Reward Model
  6. 2026Re3Cap: Retrieval-Guided Refinement for Image Captioning Enhancement via Reinforcement Learning
  7. 2024Distinctive Image Captioning: Leveraging Ground Truth Captions in CLIP Guided Reinforcement Learning
  8. 2020Learning Compact Reward for Image Captioning