mlsanityData to 7 Oct 2026

Motion-Appearance Co-Memory Networks for Video Question Answering

arXiv 1803.10906 ↗First posted 29 Mar 2018cs.CVcited by 95

Not on the map. Why?

Abstract (arXiv)

Video Question Answering (QA) is an important task in understanding video temporal structure. We observe that there are three unique attributes of video QA compared with image QA: (1) it deals with long sequences of images containing richer information not only in quantity but also in variety; (2) motion and appearance information are usually correlated with each other and able to provide useful attention cues to the other; (3) different questions require different number of frames to infer the answer. Based these observations, we propose a motion-appearance comemory network for video QA. Our networks are built on concepts from Dynamic Memory Network (DMN) and introduces new mechanisms for video QA. Specifically, there are three salient aspects: (1) a co-memory attention mechanism that utilizes cues from both motion and appearance to generate attention; (2) a temporal conv-deconv network to generate multi-level contextual facts; (3) a dynamic fact ensemble method to construct temporal representation dynamically for different questions. We evaluate our method on TGIF-QA dataset, and the results outperform state-of-the-art significantly on all four tasks of TGIF-QA.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 25; 2023: 22; 2024: 15; 2025: 8; 2026: 2.

95 papers.

Sort:

By area:

  1. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  2. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  3. 2022Zero-Shot Video Question Answering via Frozen Bidirectional Language Modelscited by 229
  4. 2021Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptionscited by 226
  5. 2021VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modelingcited by 187
  6. 2021Align and Prompt: Video-and-Language Pre-training with Entity Promptscited by 149
  7. 2025EgoLife: Towards Egocentric Life Assistantcited by 86
  8. 2022Video Question Answering: Datasets, Algorithms and Challengescited by 84
  9. 2022An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modelingcited by 74
  10. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  11. 2021Video as Conditional Graph Hierarchy for Multi-Granular Question Answeringcited by 68
  12. 2022Invariant Grounding for Video Question Answeringcited by 68
38 more
  1. 2022Video Graph Transformer for Video Question Answeringcited by 63
  2. 2022Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answeringcited by 57
  3. 2021Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answeringcited by 56
  4. 2021Natural Language Video Localization: A Revisit in Span-based Question Answering Frameworkcited by 55
  5. 2022MIST: Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answeringcited by 55
  6. 2022From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answeringcited by 51
  7. 2021Temporal Memory Attention for Video Semantic Segmentationcited by 49
  8. 2024Sports-QA: A Large-Scale Video Question Answering Benchmark for Complex and Professional Sportscited by 41
  9. 2022Temporal Sentence Grounding in Videos: A Survey and Future Directionscited by 37
  10. 2023Zero-Shot Video Question Answering with Procedural Programscited by 37
  11. 2021DualVGR: A Dual-Visual Graph Reasoning Unit for Video Question Answeringcited by 35
  12. 2023Contrastive Video Question Answering via Video Graph Transformercited by 32
  13. 2022Learning to Answer Visual Questions from Web Videoscited by 30
  14. 2021Hierarchical Object-oriented Spatio-Temporal Reasoning for Video Question Answeringcited by 29
  15. 2022(2.5+1)D Spatio-Temporal Scene Graphs for Video Question Answeringcited by 29
  16. 2021Attend What You Need: Motion-Appearance Synergistic Networks for Video Question Answeringcited by 26
  17. 2022Equivariant and Invariant Grounding for Video Question Answeringcited by 22
  18. 2023Tem-adapter: Adapting Image-Text Pretraining for Video Question Answercited by 22
  19. 2023Discovering Spatio-Temporal Rationales for Video Question Answeringcited by 21
  20. 2022Locate before Answering: Answer Guided Question Localization for Video Question Answeringcited by 19
  21. 2021CoCo-BERT: Improving Video-Language Pre-training with Contrastive Cross-modal Matching and Denoisingcited by 17
  22. 2022Video Question Answering with Iterative Video-Text Co-Tokenizationcited by 16
  23. 2023Visual Causal Scene Refinement for Video Question Answeringcited by 16
  24. 2023Learning Situation Hyper-Graphs for Video Question Answeringcited by 15
  25. 2023ANetQA: A Large-scale Benchmark for Fine-grained Compositional Reasoning over Untrimmed Videoscited by 15
  26. 2022AOE-Net: Entities Interactions Modeling with Adaptive Attention Mechanism for Temporal Action Proposals Generationcited by 14
  27. 2024Weakly Supervised Gaussian Contrastive Grounding with Large Multimodal Models for Video Question Answeringcited by 14
  28. 2024CausalChaos! Dataset for Comprehensive Causal Action Question Answering Over Longer Causal Chains Grounded in Dynamic Visual Scenescited by 13
  29. 2021Recent Advances in Video Question Answering: A Review of Datasets and Methodscited by 12
  30. 2023Simple Baselines for Interactive Video Retrieval with Questions and Answerscited by 12
  31. 2021VGNMN: Video-grounded Neural Module Network to Video-Grounded Language Taskscited by 11
  32. 2021AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposals Generationcited by 11
  33. 2022Multilevel Hierarchical Network with Multiscale Sampling for Video Question Answeringcited by 11
  34. 2022Structured Two-stream Attention Network for Video Question Answeringcited by 9
  35. 2022WildQA: In-the-Wild Video Question Answeringcited by 9
  36. 2024Video Question Answering for People with Visual Impairments Using an Egocentric 360-Degree Cameracited by 9
  37. 2024Align and Aggregate: Compositional Reasoning with Video Alignment and Answer Aggregation for Video Question-Answeringcited by 9
  38. 2024Glance and Focus: Memory Prompting for Multi-Event Video Question Answeringcited by 8

50 most cited of 95 All 95 (JSON)