mlsanityData to 7 Oct 2026

LiVLR: A Lightweight Visual-Linguistic Reasoning Framework for Video Question Answering

arXiv 2111.14547 ↗First posted 29 Nov 2021cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Video Question Answering (VideoQA), aiming to correctly answer the given question based on understanding multi-modal video content, is challenging due to the rich video content. From the perspective of video understanding, a good VideoQA framework needs to understand the video content at different semantic levels and flexibly integrate the diverse video content to distill question-related content. To this end, we propose a Lightweight Visual-Linguistic Reasoning framework named LiVLR. Specifically, LiVLR first utilizes the graph-based Visual and Linguistic Encoders to obtain multi-grained visual and linguistic representations. Subsequently, the obtained representations are integrated with the devised Diversity-aware Visual-Linguistic Reasoning module (DaVL). The DaVL considers the difference between the different types of representations and can flexibly adjust the importance of different types of representations when generating the question-related joint representation, which is an effective and general representation integration method. The proposed LiVLR is lightweight and shows its performance advantage on two VideoQA benchmarks, MRSVTT-QA and KnowIT VQA. Extensive ablation studies demonstrate the effectiveness of LiVLR key components.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 55 of its 79 references that are in the map. First posted on arXiv 29 Nov 2021.

Builds on

55 of 79 references.

References · 55

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2017Graph Attention Networkscited by 8,854
  7. 2017Mask R-CNNcited by 7,259
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  9. 2017Inductive Representation Learning on Large Graphscited by 6,284
  10. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  11. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  12. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
38 more
  1. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  2. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  3. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  4. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  5. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  6. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  7. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  8. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  9. 2017A simple neural network module for relational reasoningcited by 345
  10. 2020Just Ask: Learning to Answer Questions from Millions of Narrated Videoscited by 248
  11. 2015Deep Convolutional Networks on Graph-Structured Datacited by 246
  12. 2018Exploring Visual Relationship for Image Captioningcited by 215
  13. 2019Visual Semantic Reasoning for Image-Text Matchingcited by 185
  14. 2020Fine-grained Video-Text Retrieval with Hierarchical Graph Reasoningcited by 180
  15. 2020Hierarchical Conditional Relation Networks for Video Question Answeringcited by 163
  16. 2019Simple BERT Models for Relation Extraction and Semantic Role Labelingcited by 154
  17. 2019TVQA+: Spatio-Temporal Grounding for Video Question Answeringcited by 153
  18. 2019Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoningcited by 149
  19. 2019Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answeringcited by 146
  20. 2017Encoding Sentences with Graph Convolutional Networks for Semantic Role Labelingcited by 137
  21. 2018Graph-Based Global Reasoning Networkscited by 120
  22. 2019Relation-Aware Graph Attention Network for Visual Question Answeringcited by 119
  23. 2018Motion-Appearance Co-Memory Networks for Video Question Answeringcited by 95
  24. 2020Location-aware Graph Convolutional Networks for Video Question Answeringcited by 80
  25. 2015Spectral Representations for Convolutional Neural Networkscited by 79
  26. 2020Noise Estimation Using Density Estimation for Self-Supervised Multimodal Learningcited by 74
  27. 2019MUREL: Multimodal Relational Reasoning for Visual Question Answeringcited by 68
  28. 2021Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answeringcited by 56
  29. 2019KnowIT VQA: Answering Knowledge-Based Questions about Videoscited by 53
  30. 2020Look Before you Speak: Visually Contextualized Utterancescited by 53
  31. 2020Mucko: Multi-Layer Cross-Modal Knowledge Reasoning for Fact-based Visual Question Answeringcited by 51
  32. 2020Multi-Modal Graph Neural Network for Joint Reasoning on Vision and Scene Textcited by 46
  33. 2018Learning Conditioned Graph Structures for Interpretable Visual Question Answeringcited by 45
  34. 2019Probabilistic Neural-symbolic Models for Interpretable Visual Question Answeringcited by 38
  35. 2020Modality Shifting Attention Network for Multi-modal Video Question Answeringcited by 38
  36. 2020Neuro-Symbolic Visual Reasoning: Disentangling "Visual" from "Reasoning"cited by 35
  37. 2021DualVGR: A Dual-Visual Graph Reasoning Unit for Video Question Answeringcited by 35
  38. 2021Attend What You Need: Motion-Appearance Synergistic Networks for Video Question Answeringcited by 26

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 1.

4 papers.

  1. 2023Long Video Understanding with Learnable Retrieval in Video-Language Modelscited by 9
  2. 2023Taking A Closer Look at Visual Relation: Unbiased Video Scene Graph Generation with Decoupled Label Learningcited by 3
  3. 2024Answering from Sure to Uncertain: Uncertainty-Aware Curriculum Learning for Video Question Answeringcited by 2
  4. 2023Video Captioning with Aggregated Features Based on Dual Graphs and Gated Fusioncited by 1