mlsanityData to 14 Oct 2026

MoReVQA: Exploring Modular Reasoning Models for Video Question Answering

arXiv 2404.06511 ↗First posted 9 Apr 2024cs.CV cs.AI cs.LGcited by 74

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

This paper addresses the task of video question answering (videoQA) via a decomposed multi-stage, modular reasoning framework. Previous modular methods have shown promise with a single planning stage ungrounded in visual content. However, through a simple and effective baseline, we find that such systems can lead to brittle behavior in practice for challenging videoQA settings. Thus, unlike traditional single-stage planning methods, we propose a multi-stage system consisting of an event parser, a grounding stage, and a final reasoning stage in conjunction with an external memory. All stages are training-free, and performed using few-shot prompting of large models, creating interpretable intermediate outputs at each stage. By decomposing the underlying planning and task complexity, our method, MoReVQA, improves over prior work on standard videoQA benchmarks (NExT-QA, iVQA, EgoSchema, ActivityNet-QA) with state-of-the-art results, and extensions to related tasks (grounded videoQA, paragraph captioning).

Key facts. Cited by 74 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 66 of its 77 references that are in the map. First posted on arXiv 9 Apr 2024.

Builds on

66 of 77 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023ViperGPT: Visual Inference via Python Execution for Reasoning14 Mar
  2. 2022Visual Programming: Compositional visual reasoning without training18 Nov
  3. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language1 Apr

Other references · 63

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  10. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  11. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  12. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
38 more
  1. 2021ViViT: A Video Vision Transformercited by 1,620
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  3. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  4. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  5. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  6. 2018Model Cards for Model Reportingcited by 1,080
  7. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  8. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  9. 2017Dense-Captioning Events in Videoscited by 793
  10. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  11. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
  12. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  13. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  14. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  15. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  16. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
  17. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 537
  18. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 447
  19. 2015Neural Module Networkscited by 376
  20. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 305
  21. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  22. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 255
  23. 2020Just Ask: Learning to Answer Questions from Millions of Narrated Videoscited by 248
  24. 2022Zero-Shot Video Question Answering via Frozen Bidirectional Language Modelscited by 229
  25. 2018End-to-End Dense Video Captioning with Masked Transformercited by 214
  26. 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 203
  27. 2021VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modelingcited by 188
  28. 2022Revisiting the "Video" in Video-Language Understandingcited by 169
  29. 2023Can I Trust Your Answer? Visually Grounded Video Question Answeringcited by 135
  30. 2021End-to-End Dense Video Captioning with Parallel Decodingcited by 133
  31. 2022Revealing Single Frame Bias for Video-and-Language Learningcited by 133
  32. 2017Inferring and Executing Programs for Visual Reasoningcited by 130
  33. 2021Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implicationscited by 128
  34. 2023PaLI-3 Vision Language Models: Smaller, Faster, Strongercited by 126
  35. 2021AGQA: A Benchmark for Compositional Spatio-Temporal Reasoningcited by 120
  36. 2022Language Models with Image Descriptors are Strong Few-Shot Video-Language Learnerscited by 117
  37. 2022End-to-end Generative Pretraining for Multimodal Video Captioningcited by 116
  38. 2022Video Question Answering: Datasets, Algorithms and Challengescited by 84

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 36; 2026: 27.

74 papers.

Sort:

By area:

  1. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 251
  2. 2024Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehensioncited by 138
  3. 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuningcited by 93
  4. 2025BOLT: Boost Large Vision-Language Model Without Training for Long-form Video Understandingcited by 75
  5. 2024Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QAcited by 68
  6. 2024DrVideo: Document Retrieval Based Long Video Understandingcited by 65
  7. 2024Enhancing Video-LLM Reasoning via Agent-of-Thoughts Distillationcited by 37
  8. 2024VideoQA in the Era of LLMs: An Empirical Studycited by 29
  9. 2025Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understandingcited by 29
  10. 2024Neptune: The Long Orbit to Benchmarking Long Video Understandingcited by 25
  11. 2025Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)cited by 23
  12. 2026Large Language Model Reasoning Failurescited by 21
38 more
  1. 2025Building a Mind Palace: Structuring Environment-Grounded Semantic Graphs for Effective Long Video Analysis with LLMscited by 19
  2. 2025Time Blindness: Why Video-Language Models Can't See What Humans Can?cited by 18
  3. 2025Video Summarization with Large Language Modelscited by 16
  4. 2025When Thinking Drifts: Evidential Grounding for Robust Video Reasoningcited by 16
  5. 2024SEAL: Semantic Attention Learning for Long Video Representationcited by 13
  6. 2025AdsQA: Towards Advertisement Video Understandingcited by 13
  7. 2025Active Perception Agent for Omnimodal Audio-Video Understandingcited by 13
  8. 2025Cross-modal Causal Relation Alignment for Video Question Groundingcited by 12
  9. 2025Divide, then Ground: Adapting Frame Selection to Query Types for Long-Form Video Understandingcited by 11
  10. 2024TOPA: Extending Large Language Models for Video Understanding via Text-Only Pre-Alignmentcited by 10
  11. 2025Threading Keyframe with Narratives: MLLMs as Strong Long Video Comprehenderscited by 10
  12. 2025MimeQA: Towards Socially-Intelligent Nonverbal Foundation Modelscited by 9
  13. 2025Chapter-Llama: Efficient Chaptering in Hour-Long Videos with LLMscited by 9
  14. 2025From Frames to Clips: Training-free Adaptive Key Clip Selection for Long-Form Video Understandingcited by 9
  15. 2025How Can Objects Help Video-Language Understanding?cited by 8
  16. 2026Think, Then Verify: A Hypothesis-Verification Multi-Agent Framework for Long Video Understandingcited by 8
  17. 2025FrameOracle: Learning What to See and How Much to See in Videoscited by 7
  18. 2026Event-Anchored Frame Selection for Effective Long-Video Understandingcited by 7
  19. 2025From Shots to Stories: LLM-Assisted Video Editing with Unified Language Representationscited by 6
  20. 2025AViLA: Asynchronous Vision-Language Agent for Streaming Multimodal Data Interactioncited by 6
  21. 2026VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAGcited by 6
  22. 2026MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answeringcited by 6
  23. 2025ENTER: Event Based Interpretable Reasoning for VideoQAcited by 5
  24. 2025ViQAgent: Zero-Shot Video Question Answering via Agent with Open-Vocabulary Grounding Validationcited by 5
  25. 2025Moment Sampling in Video LLMs for Long-Form Video QAcited by 5
  26. 2026OmniRAG-Agent: Agentic Omnimodal Reasoning for Low-Resource Long Audio-Video Question Answeringcited by 5
  27. 2026Video Understanding: From Geometry and Semantics to Unified Modelscited by 5
  28. 2026Soap2Soap: Long Cinematic Video Remaking via Multi-Agent Collaborationcited by 5
  29. 2024Episodic Memory Verbalization using Hierarchical Representations of Life-Long Robot Experiencecited by 4
  30. 2025EmoSign: A Multimodal Dataset for Understanding Emotions in American Sign Languagecited by 4
  31. 2025A Neurosymbolic Agent System for Compositional Visual Reasoningcited by 4
  32. 2025MUPA: Towards Multi-Path Agentic Reasoning for Grounded Video Question Answeringcited by 4
  33. 2025LeAdQA: LLM-Driven Context-Aware Temporal Grounding for Video Question Answeringcited by 4
  34. 2025Bridging Vision Language Models and Symbolic Grounding for Video Question Answeringcited by 4
  35. 2025Neuro Symbolic Knowledge Reasoning for Procedural Video Question Answeringcited by 3
  36. 2025Looking Beyond Visible Cues: Implicit Video Question Answering via Dual-Clue Reasoningcited by 3
  37. 2025Empowering Multimodal LLMs with External Tools: A Comprehensive Surveycited by 2
  38. 2026A Multi-Agent Perception-Action Alliance for Efficient Long Video Reasoningcited by 2

50 most cited of 74 All 74 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025ReasVQA: Advancing VideoQA with Imperfect Reasoning Process
  2. 2024Question-Instructed Visual Descriptions for Zero-Shot Video Question Answering
  3. 2024Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answering
  4. 2025Team of One: Cracking Complex Video QA with Model Synergy
  5. 2023A Simple LLM Framework for Long-Range Video Question-Answering
  6. 2025Commonsense Video Question Answering through Video-Grounded Entailment Tree Reasoning
  7. 2024Perceive, Query & Reason: Enhancing Video QA with Question-Guided Temporal Queries
  8. 2025VideoMultiAgents: A Multi-Agent Framework for Video Question Answering