mlsanityData to 7 Oct 2026

Object Relational Graph with Teacher-Recommended Learning for Video Captioning

arXiv 2002.11566 ↗First posted 26 Feb 2020cs.CV cs.CLcited by 106

Not on the map. Why?

Abstract (arXiv)

Taking full advantage of the information from both vision and language is critical for the video captioning task. Existing models lack adequate visual representation due to the neglect of interaction between object, and sufficient training for content-related words due to long-tailed problems. In this paper, we propose a complete video captioning system including both a novel model and an effective training strategy. Specifically, we propose an object relational graph (ORG) based encoder, which captures more detailed interaction features to enrich visual representation. Meanwhile, we design a teacher-recommended learning (TRL) method to make full use of the successful external language model (ELM) to integrate the abundant linguistic knowledge into the caption model. The ELM generates more semantically similar word proposals which extend the ground-truth words used for training to deal with the long-tailed problem. Experimental evaluations on three benchmarks: MSVD, MSR-VTT and VATEX show the proposed ORG-TRL system achieves state-of-the-art performance. Extensive ablation studies and visualizations illustrate the effectiveness of our system.

Key facts. Cited by 106 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 32; 2023: 27; 2024: 14; 2025: 5; 2026: 2.

106 papers.

Sort:

By area:

  1. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  2. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 536
  3. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 269
  4. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  5. 2023Any-to-Any Generation via Composable Diffusioncited by 203
  6. 2021SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioningcited by 161
  7. 2023VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Datasetcited by 138
  8. 2021Uni-Perceiver: Pre-training Unified Architecture for Generic Perception for Zero-shot and Few-shot Taskscited by 124
  9. 2022End-to-end Generative Pretraining for Multimodal Video Captioningcited by 115
  10. 2022Self-Supervised Learning of Object Parts for Semantic Segmentationcited by 90
  11. 2021CLIP4Caption: CLIP for Video Captioncited by 88
  12. 2022Learning Audio-Video Modalities from Image Captionscited by 80
38 more
  1. 2023ADAPT: Action-aware Driving Caption Transformercited by 78
  2. 2022A Survey on Long-Tailed Visual Recognitioncited by 75
  3. 2022VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captionerscited by 69
  4. 2023CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generationcited by 63
  5. 2021Joint Inductive and Transductive Learning for Video Object Segmentationcited by 61
  6. 2022Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEscited by 58
  7. 2022Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representationscited by 57
  8. 2022A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspectivecited by 56
  9. 2021Exploring Sequence Feature Alignment for Domain Adaptive Detection Transformerscited by 49
  10. 2021Open-book Video Captioning with Retrieve-Copy-Generate Networkcited by 43
  11. 2021Learning Conditional Knowledge Distillation for Degraded-Reference Image Quality Assessmentcited by 41
  12. 2023VidChapters-7M: Video Chapters at Scalecited by 39
  13. 2021Disentangle Your Dense Object Detectorcited by 37
  14. 2024Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectivescited by 37
  15. 2021EMScore: Evaluating Video Captioning via Coarse-Grained and Fine-Grained Embedding Matchingcited by 36
  16. 2023GPT4Video: A Unified Multimodal Large Language Model for lnstruction-Followed Understanding and Safety-Aware Generationcited by 36
  17. 2021Semantic Grouping Network for Video Captioningcited by 33
  18. 2021Self-Supervised Visual Representations Learning by Contrastive Mask Predictioncited by 31
  19. 2021Hierarchical Modular Network for Video Captioningcited by 31
  20. 2022Visual Abductive Reasoningcited by 30
  21. 2023GOAL: A Challenging Knowledge-grounded Video Captioning Benchmark for Real-time Soccer Commentary Generationcited by 28
  22. 2023A Review of Deep Learning for Video Captioningcited by 28
  23. 2023Bootstrapping Vision-Language Learning with Decoupled Language Pre-trainingcited by 28
  24. 2023Text with Knowledge Graph Augmented Transformer for Video Captioningcited by 27
  25. 2022RCL: Recurrent Continuous Localization for Temporal Action Detectioncited by 25
  26. 2023MaMMUT: A Simple Architecture for Joint Learning for MultiModal Taskscited by 24
  27. 2024MECD: Unlocking Multi-Event Causal Discovery in Video Reasoningcited by 23
  28. 2021Towards Diverse Paragraph Captioning for Untrimmed Videoscited by 22
  29. 2024OmniVid: A Generative Framework for Universal Video Understandingcited by 22
  30. 2023Accurate and Fast Compressed Video Captioningcited by 21
  31. 2021O2NA: An Object-Oriented Non-Autoregressive Approach for Controllable Video Captioningcited by 18
  32. 2022Rethinking Data Augmentation for Robust Visual Question Answeringcited by 18
  33. 2022Respecting Transfer Gap in Knowledge Distillationcited by 18
  34. 2022Refined Semantic Enhancement towards Frequency Diffusion for Video Captioningcited by 18
  35. 2023Accommodating Audio Modality in CLIP for Multimodal Processingcited by 18
  36. 2024ModaVerse: Efficiently Transforming Modalities with LLMscited by 18
  37. 2021DnS: Distill-and-Select for Efficient and Accurate Video Indexing and Retrievalcited by 17
  38. 2021CoCo-BERT: Improving Video-Language Pre-training with Contrastive Cross-modal Matching and Denoisingcited by 17

50 most cited of 106 All 106 (JSON)