mlsanityData to 14 Oct 2026

Video Captioning via Hierarchical Reinforcement Learning

arXiv 1711.11135 ↗First posted 29 Nov 2017cs.CV cs.AI cs.CLcited by 47

Not on the map. Why?

Abstract (arXiv)

Video captioning is the task of automatically generating a textual description of the actions in a video. Although previous work (e.g. sequence-to-sequence model) has shown promising results in abstracting a coarse description of a short video, it is still very challenging to caption a video containing multiple fine-grained actions with a detailed description. This paper aims to address the challenge by proposing a novel hierarchical reinforcement learning framework for video captioning, where a high-level Manager module learns to design sub-goals and a low-level Worker module recognizes the primitive actions to fulfill the sub-goal. With this compositional framework to reinforce video captioning at different levels, our approach significantly outperforms all the baseline methods on a newly introduced large-scale dataset for fine-grained video captioning. Furthermore, our non-ensemble model has already achieved the state-of-the-art results on the widely-used MSR-VTT dataset.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2018: 1; 2019: 0; 2020: 1; 2021: 12; 2022: 11; 2023: 12; 2024: 5; 2025: 5.

47 papers.

Sort:

By area:

  1. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  2. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
  3. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  4. 2024Frame-Voyager: Learning to Query Frames for Video Large Language Modelscited by 83
  5. 2022VLMbench: A Compositional Benchmark for Vision-and-Language Manipulationcited by 68
  6. 2021Procedure Planning in Instructional Videos via Contextual Modeling and Model-based Policy Learningcited by 53
  7. 2023VidChapters-7M: Video Chapters at Scalecited by 39
  8. 2023VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Viewscited by 38
  9. 2022A Review on Methods and Applications in Multimodal Deep Learningcited by 29
  10. 2023A Review of Deep Learning for Video Captioningcited by 28
  11. 2023Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applicationscited by 27
  12. 2021Recent Advances and Trends in Multimodal Deep Learning: A Reviewcited by 26
35 more
  1. 2023ScribbleVC: Scribble-supervised Medical Image Segmentation with Vision-Class Embeddingcited by 23
  2. 2021Towards Diverse Paragraph Captioning for Untrimmed Videoscited by 22
  3. 2021ClawCraneNet: Leveraging Object-level Relation for Text-based Video Segmentationcited by 21
  4. 2023Models See Hallucinations: Evaluating the Factuality in Video Captioningcited by 19
  5. 2022GL-RG: Global-Local Representation Granularity for Video Captioningcited by 16
  6. 2021Relational Graph Learning for Grounded Video Description Generationcited by 12
  7. 2024ALaRM: Align Language Models via Hierarchical Rewards Modelingcited by 11
  8. 2024Question-Answering Dense Video Eventscited by 11
  9. 2025VideoMiner: Iteratively Grounding Key Frames of Hour-Long Videos via Tree-based Group Relative Policy Optimizationcited by 11
  10. 2021Controllable Video Captioning with an Exemplar Sentencecited by 9
  11. 2024Movie101v2: Improved Movie Narration Benchmarkcited by 9
  12. 2022Planning Irregular Object Packing via Hierarchical Reinforcement Learningcited by 8
  13. 2021Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translationcited by 7
  14. 2025Transferability of Adversarial Attacks in Video-based MLLMs: A Cross-modal Image-to-Video Approachcited by 7
  15. 2025IntentVCNet: Bridging Spatio-Temporal Gaps for Intention-Oriented Controllable Video Captioningcited by 7
  16. 2021A Survey on Natural Language Video Localizationcited by 6
  17. 2021A Comprehensive Review of the Video-to-Text Problemcited by 5
  18. 2023Zero-Shot Dense Video Captioning by Jointly Optimizing Text and Momentcited by 4
  19. 2020Nested-Wasserstein Self-Imitation Learning for Sequence Generationcited by 3
  20. 2022Taking an Emotional Look at Video Paragraph Captioningcited by 3
  21. 2022Video Captioning: a comparative review of where we are and which could be the routecited by 3
  22. 2021DVCFlow: Modeling Information Flow Towards Human-like Video Captioningcited by 2
  23. 2022Relational Future Captioning Model for Explaining Likely Collisions in Daily Taskscited by 2
  24. 2021The Role of the Input in Natural Language Video Descriptioncited by 1
  25. 2021Where to Look: A Unified Attention Model for Visual Recognition with Reinforcement Learningcited by 1
  26. 2022METEOR Guided Divergence for Video Captioningcited by 1
  27. 2023Implicit and Explicit Commonsense for Multi-sentence Video Captioningcited by 1
  28. 2023Nepali Video Captioning using CNN-RNN Architecturecited by 1
  29. 2024Robust Model Based Reinforcement Learning Using L1 Adaptive Controlcited by 1
  30. 2025Dense Video Captioning using Graph-based Sentence Summarizationcited by 1
  31. 2022An Integrated Approach for Video Captioning and Applicationscited by 0
  32. 2022Bringing Image Scene Structure to Video via Frame-Clip Consistency of Object Tokenscited by 0
  33. 2023Multi-modal Instance Refinement for Cross-domain Action Recognitioncited by 0
  34. 2023Attention Based Encoder Decoder Model for Video Captioning in Nepali (2023)cited by 0
  35. 2025Show, Tell and Summarize: Dense Video Captioning Using Visual Cue Aided Sentence Summarizationcited by 0