mlsanityData to 7 Oct 2026

Spatio-Temporal Graph for Video Captioning with Knowledge Distillation

arXiv 2003.13942 ↗First posted 31 Mar 2020cs.CVcited by 109

Not on the map. Why?

Abstract (arXiv)

Video captioning is a challenging task that requires a deep understanding of visual scenes. State-of-the-art methods generate captions using either scene-level or object-level information but without explicitly modeling object interactions. Thus, they often fail to make visually grounded predictions, and are sensitive to spurious correlations. In this paper, we propose a novel spatio-temporal graph model for video captioning that exploits object interactions in space and time. Our model builds interpretable links and is able to provide explicit visual grounding. To avoid unstable performance caused by the variable number of objects, we further propose an object-aware knowledge distillation mechanism, in which local object information is used to regularize global scene features. We demonstrate the efficacy of our approach through extensive experiments on two benchmarks, showing our approach yields competitive performance with interpretable predictions.

Key facts. Cited by 109 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 29; 2022: 33; 2023: 19; 2024: 18; 2025: 7; 2026: 3.

109 papers.

Sort:

By area:

  1. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 536
  2. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  3. 2022Multimodal Learning with Transformers: A Surveycited by 399
  4. 2021SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioningcited by 161
  5. 2021Ego-Exo: Transferring Visual Representations from Third-person to First-person Videoscited by 98
  6. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 93
  7. 2023ADAPT: Action-aware Driving Caption Transformercited by 78
  8. 2022Knowledge Distillation via the Target-aware Transformercited by 70
  9. 2022Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representationscited by 57
  10. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  11. 2022A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspectivecited by 56
  12. 2022RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detectioncited by 54
38 more
  1. 2021Activity Graph Transformer for Temporal Action Localizationcited by 48
  2. 2021End-to-end Multi-modal Video Temporal Groundingcited by 47
  3. 2022The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillationcited by 44
  4. 2021Open-book Video Captioning with Retrieve-Copy-Generate Networkcited by 43
  5. 2022TiG-BEV: Multi-view BEV 3D Object Detection via Target Inner-Geometry Learningcited by 37
  6. 2024Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectivescited by 37
  7. 2021Semantic Grouping Network for Video Captioningcited by 33
  8. 2021Hierarchical Modular Network for Video Captioningcited by 31
  9. 2022Visual Abductive Reasoningcited by 30
  10. 2021Hierarchical Object-oriented Spatio-Temporal Reasoning for Video Question Answeringcited by 29
  11. 2022(2.5+1)D Spatio-Temporal Scene Graphs for Video Question Answeringcited by 29
  12. 2023A Review of Deep Learning for Video Captioningcited by 28
  13. 2023Text with Knowledge Graph Augmented Transformer for Video Captioningcited by 27
  14. 2023ViLA: Efficient Video-Language Alignment for Video Question Answeringcited by 23
  15. 2021Towards Diverse Paragraph Captioning for Untrimmed Videoscited by 22
  16. 2024OmniVid: A Generative Framework for Universal Video Understandingcited by 22
  17. 2023Accurate and Fast Compressed Video Captioningcited by 21
  18. 2023Models See Hallucinations: Evaluating the Factuality in Video Captioningcited by 19
  19. 2024Narrative Action Evaluation with Prompt-Guided Multimodal Interactioncited by 19
  20. 2021O2NA: An Object-Oriented Non-Autoregressive Approach for Controllable Video Captioningcited by 18
  21. 2021IntentVizor: Towards Generic Query Guided Interactive Video Summarizationcited by 18
  22. 2022Rethinking Data Augmentation for Robust Visual Question Answeringcited by 18
  23. 2022Respecting Transfer Gap in Knowledge Distillationcited by 18
  24. 2022Refined Semantic Enhancement towards Frequency Diffusion for Video Captioningcited by 18
  25. 2021DnS: Distill-and-Select for Efficient and Accurate Video Indexing and Retrievalcited by 17
  26. 2023Fine-grained Audible Video Descriptioncited by 17
  27. 2022GL-RG: Global-Local Representation Granularity for Video Captioningcited by 16
  28. 2024TimeRefine: Temporal Grounding with Time Refining Video LLMcited by 15
  29. 2022Aligning Source Visual and Target Language Domains for Unpaired Video Captioningcited by 14
  30. 2023Computation-efficient Deep Learning for Computer Vision: A Surveycited by 14
  31. 2022Sports Video Analysis on Large-Scale Datacited by 13
  32. 2024A Modular Approach for Multimodal Summarization of TV Showscited by 13
  33. 2021Finding the Right Moment: Human-Assisted Trailer Creation via Task Compositioncited by 12
  34. 2025Enrich and Detect: Video Temporal Grounding with Multimodal LLMscited by 12
  35. 2021CLIP Meets Video Captioning: Concept-Aware Representation Learning Does Mattercited by 11
  36. 2022Non-Parallel Text Style Transfer with Self-Parallel Supervisioncited by 11
  37. 2022Video Dialog as Conversation about Objects Living in Space-Timecited by 11
  38. 2024NarrativeBridge: Enhancing Video Captioning with Causal-Temporal Narrativecited by 10

50 most cited of 109 All 109 (JSON)