mlsanityData to 7 Oct 2026

From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answering

arXiv 2205.14895 ↗First posted 30 May 2022cs.CV cs.CL cs.MMcited by 51

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Video understanding has achieved great success in representation learning, such as video caption, video object grounding, and video descriptive question-answer. However, current methods still struggle on video reasoning, including evidence reasoning and commonsense reasoning. To facilitate deeper video understanding towards video reasoning, we present the task of Causal-VidQA, which includes four types of questions ranging from scene description (description) to evidence reasoning (explanation) and commonsense reasoning (prediction and counterfactual). For commonsense reasoning, we set up a two-step solution by answering the question and providing a proper reason. Through extensive experiments on existing VideoQA methods, we find that the state-of-the-art methods are strong in descriptions but weak in reasoning. We hope that Causal-VidQA can guide the research of video understanding from representation learning to deeper reasoning. The dataset and related resources are available at urlhttps://github.com/bcmi/Causal-VidQA.git.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 40 of its 57 references that are in the map. First posted on arXiv 30 May 2022.

Builds on

40 of 57 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  6. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  8. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  9. 2015VQA: Visual Question Answeringcited by 2,489
  10. 2018Deep contextualized word representationscited by 2,285
  11. 2017The Kinetics Human Action Video Datasetcited by 1,894
  12. 2018SlowFast Networks for Video Recognitioncited by 1,862
28 more
  1. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  2. 2017Dense-Captioning Events in Videoscited by 789
  3. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  4. 2017Localizing Moments in Video with Natural Languagecited by 660
  5. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559
  6. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  7. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  8. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
  9. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  10. 2019Video Instance Segmentationcited by 391
  11. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  12. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
  13. 2018End-to-End Dense Video Captioning with Masked Transformercited by 214
  14. 2020Hierarchical Conditional Relation Networks for Video Question Answeringcited by 163
  15. 2019Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answeringcited by 146
  16. 2020Collaborative Video Object Segmentation by Foreground-Background Integrationcited by 142
  17. 2016TGIF: A New Dataset and Benchmark on Animated GIF Descriptioncited by 124
  18. 2018Motion-Appearance Co-Memory Networks for Video Question Answeringcited by 95
  19. 2020Location-aware Graph Convolutional Networks for Video Question Answeringcited by 80
  20. 2021Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answeringcited by 56
  21. 2016MarioQA: Answering Questions by Watching Gameplay Videoscited by 42
  22. 2016A dataset and exploration of models for understanding video data through fill-in-the-blank question-answeringcited by 41
  23. 2020Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioningcited by 36
  24. 2020On Modality Bias in the TVQA Datasetcited by 24
  25. 2019Progressive Attention Memory Network for Movie Story Question Answeringcited by 21
  26. 2018Multimodal Dual Attention Memory for Video Story Question Answeringcited by 20
  27. 2020Visual Relation Grounding in Videoscited by 19
  28. 2018A Dataset and Architecture for Visual Reasoning with a Working Memorycited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 14; 2025: 16; 2026: 10.

51 papers.

Sort:

By area:

  1. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognitioncited by 151
  2. 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 144
  3. 2024Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QAcited by 68
  4. 2025V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoningcited by 66
  5. 2024Vision-and-Language Navigation via Causal Learningcited by 51
  6. 2023Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learningcited by 45
  7. 2025CausalVQA: A Physically Grounded Causal Reasoning Benchmark for Video Modelscited by 35
  8. 2023Contrastive Video Question Answering via Video Graph Transformercited by 32
  9. 2025Foundation Models for Spatio-Temporal Data Science: A Tutorial and Surveycited by 23
  10. 2023Discovering Spatio-Temporal Rationales for Video Question Answeringcited by 21
  11. 2025Robust Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoningcited by 20
  12. 2025Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Markscited by 17
38 more
  1. 2023Visual Causal Scene Refinement for Video Question Answeringcited by 16
  2. 2024Referring Atomic Video Action Recognitioncited by 16
  3. 2024Weakly Supervised Gaussian Contrastive Grounding with Large Multimodal Models for Video Question Answeringcited by 14
  4. 2024CausalChaos! Dataset for Comprehensive Causal Action Question Answering Over Longer Causal Chains Grounded in Dynamic Visual Scenescited by 13
  5. 2025Dr.V: A Hierarchical Perception-Temporal-Cognition Framework to Diagnose Video Hallucination by Fine-grained Spatial-Temporal Groundingcited by 13
  6. 2024SEAL: Semantic Attention Learning for Long Video Representationcited by 12
  7. 2023A Multi-Modal Context Reasoning Approach for Conditional Inference on Joint Textual and Visual Cluescited by 11
  8. 2024Align and Aggregate: Compositional Reasoning with Video Alignment and Answer Aggregation for Video Question-Answeringcited by 9
  9. 2024Black Swan: Abductive and Defeasible Video Reasoning in Unpredictable Eventscited by 9
  10. 2024MAMA: Meta-optimized Angular Margin Contrastive Framework for Video-Language Representation Learningcited by 8
  11. 2025Advancing Egocentric Video Question Answering with Multimodal Large Language Modelscited by 8
  12. 2023Redundancy-aware Transformer for Video Question Answeringcited by 7
  13. 2023Language Models are Causal Knowledge Extractors for Zero-shot Video Question Answeringcited by 6
  14. 2025What Changed and What Could Have Changed? State-Change Counterfactuals for Procedure-Aware Video Representation Learningcited by 6
  15. 2025R^3-VQA: "Read the Room" by Video Social Reasoningcited by 6
  16. 2025Causal-LLaVA: Causal Disentanglement for Mitigating Hallucination in Multimodal Large Language Modelscited by 4
  17. 2023ATM: Action Temporality Modeling for Video Question Answeringcited by 3
  18. 2025Looking Beyond Visible Cues: Implicit Video Question Answering via Dual-Clue Reasoningcited by 3
  19. 2025Transformer-Based Spatial-Temporal Counterfactual Outcomes Estimationcited by 3
  20. 2026YoCausal: How Far is Video Generation from World Model? A Causality Perspectivecited by 3
  21. 2025Video-VoT-R1: An efficient video inference model integrating image packing and AoE architecturecited by 2
  22. 2026It's a Matter of Time: Three Lessons on Long-Term Motion for Perceptioncited by 2
  23. 2026VEBench:Benchmarking Large Multimodal Models for Real-World Video Editingcited by 2
  24. 2023What If the TV Was Off? Examining Counterfactual Reasoning Abilities of Multi-modal Language Modelscited by 1
  25. 2024Investigating Video Reasoning Capability of Large Language Models with Tropes in Moviescited by 1
  26. 2025Robust Egocentric Referring Video Object Segmentation via Dual-Modal Causal Interventioncited by 1
  27. 2026KARMA-MV: A Benchmark for Causal Question Answering on Music Videoscited by 1
  28. 2026Perception Without Engagement: Dissecting the Causal Discovery Deficit in LMMscited by 1
  29. 2023Apollo: Zero-shot MultiModal Reasoning with Multiple Expertscited by 0
  30. 2024Causality-based Cross-Modal Representation Learning for Vision-and-Language Navigationcited by 0
  31. 2024Action Reimagined: Text-to-Pose Video Editing for Dynamic Human Actionscited by 0
  32. 2024Foundation Models and Adaptive Feature Selection: A Synergistic Approach to Video Question Answeringcited by 0
  33. 2025VUDG: A Dataset for Video Understanding Domain Generalizationcited by 0
  34. 2025ChainReaction: Causal Chain-Guided Reasoning for Modular and Explainable Causal-Why Video Question Answeringcited by 0
  35. 2026SVI-Bench: A Dynamic Microworld for Strategic Video Intelligencecited by 0
  36. 2026EgoSafe: A First-Person Mobile-Captured Benchmark for Visual Safety Understandingcited by 0
  37. 2026Action- and Language-Conditioned Video Assessment for Embodied Controlcited by 0
  38. 2026IntentQA: Intent Question Answering in Videos by Cognitive Context Reasoningcited by 0

50 most cited of 51 All 51 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Commonsense Video Question Answering through Video-Grounded Entailment Tree Reasoning
  2. 2025ReasVQA: Advancing VideoQA with Imperfect Reasoning Process
  3. 2026UpstreamQA: A Modular Framework for Explicit Reasoning on Video Question Answering Tasks
  4. 2025VidBridge-R1: Bridging QA and Captioning for RL-based Video Understanding Models with Intermediate Proxy Tasks
  5. 2022Watching the News: Towards VideoQA Models that can Read
  6. 2022Joint Answering and Explanation for Visual Commonsense Reasoning
  7. 2022Invariant Grounding for Video Question Answering
  8. 2024MoReVQA: Exploring Modular Reasoning Models for Video Question Answering