# Leveraging Static Relationships for Intra-Type and Inter-Type Message Passing in Video Question Answering

arXiv: https://arxiv.org/abs/2504.02417 · HTML: https://mlsanity.com/paper/2504.02417/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 42 references that are in the map.
- First posted on arXiv 3 Apr 2025.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions](https://mlsanity.com/paper/2105.08276/) (2021)
- [Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling](https://mlsanity.com/paper/2102.06183/) (2021)
- [All in One: Exploring Unified Video-Language Pre-training](https://mlsanity.com/paper/2203.07303/) (2022)
- [Revisiting the "Video" in Video-Language Understanding](https://mlsanity.com/paper/2206.01720/) (2022)
- [GPS-Net: Graph Property Sensing Network for Scene Graph Generation](https://mlsanity.com/paper/2003.12962/) (2020)
- [Hierarchical Conditional Relation Networks for Video Question Answering](https://mlsanity.com/paper/2002.10698/) (2020)
- [Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering](https://mlsanity.com/paper/1904.04357/) (2019)
- [Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2104.00308/) (2021)
- [Recovering the Unbiased Scene Graphs from the Biased Ones](https://mlsanity.com/paper/2107.02112/) (2021)
- [Video as Conditional Graph Hierarchy for Multi-Granular Question Answering](https://mlsanity.com/paper/2112.06197/) (2021)
- [Invariant Grounding for Video Question Answering](https://mlsanity.com/paper/2206.02349/) (2022)
- [Learning to Generate Scene Graph from Natural Language Supervision](https://mlsanity.com/paper/2109.02227/) (2021)
- [Language-Conditioned Graph Networks for Relational Reasoning](https://mlsanity.com/paper/1905.04405/) (2019)
- [MIST: Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answering](https://mlsanity.com/paper/2212.09522/) (2022)
- [MMFT-BERT: Multimodal Fusion Transformer with BERT Encodings for Visual Question Answering](https://mlsanity.com/paper/2010.14095/) (2020)
- [(2.5+1)D Spatio-Temporal Scene Graphs for Video Question Answering](https://mlsanity.com/paper/2202.09277/) (2022)
- [Hierarchical Object-oriented Spatio-Temporal Reasoning for Video Question Answering](https://mlsanity.com/paper/2106.13432/) (2021)
- [Unbiased Heterogeneous Scene Graph Generation with Relation-aware Message Passing Neural Network](https://mlsanity.com/paper/2212.00443/) (2022)
- [Learning Situation Hyper-Graphs for Video Question Answering](https://mlsanity.com/paper/2304.08682/) (2023)
- [ANetQA: A Large-scale Benchmark for Fine-grained Compositional Reasoning over Untrimmed Videos](https://mlsanity.com/paper/2305.02519/) (2023)
- [Label Semantic Knowledge Distillation for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2208.03763/) (2022)
- [Dual-branch Hybrid Learning Network for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2207.07913/) (2022)
- [Glance and Focus: Memory Prompting for Multi-Event Video Question Answering](https://mlsanity.com/paper/2401.01529/) (2024)
- [Importance Weighted Structure Learning for Scene Graph Generation](https://mlsanity.com/paper/2205.07017/) (2022)
- [Efficient End-to-End Video Question Answering with Pyramidal Multimodal Transformer](https://mlsanity.com/paper/2302.02136/) (2023)
- [ATM: Action Temporality Modeling for Video Question Answering](https://mlsanity.com/paper/2309.02290/) (2023)

## Abstract (arXiv)

Video Question Answering (VideoQA) is an important research direction in the field of artificial intelligence, enabling machines to understand video content and perform reasoning and answering based on natural language questions. Although methods based on static relationship reasoning have made certain progress, there are still deficiencies in the accuracy of static relationship recognition and representation, and they have not fully utilized the static relationship information in videos for in-depth reasoning and analysis. Therefore, this paper proposes a reasoning method for intra-type and inter-type message passing based on static relationships. This method constructs a dual graph for intra-type message passing reasoning and builds a heterogeneous graph based on static relationships for inter-type message passing reasoning. The intra-type message passing reasoning model captures the neighborhood information of targets and relationships related to the question in the dual graph, updating the dual graph to obtain intra-type clues for answering the question. The inter-type message passing reasoning model captures the neighborhood information of targets and relationships from different categories related to the question in the heterogeneous graph, updating the heterogeneous graph to obtain inter-type clues for answering the question. Finally, the answers are inferred by combining the intra-type and inter-type clues based on static relationships. Experimental results on the ANetQA and Next-QA datasets demonstrate the effectiveness of this method.