# MCQA: Multimodal Co-attention Based Network for Question Answering

arXiv: https://arxiv.org/abs/2004.12238 · HTML: https://mlsanity.com/paper/2004.12238/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2020.

## Built on by (most cited first, 7 of 7)

- [Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions](https://mlsanity.com/paper/2107.13782/) (2021)
- [FedMEKT: Distillation-based Embedding Knowledge Transfer for Multimodal Federated Learning](https://mlsanity.com/paper/2307.13214/) (2023)
- [MuRAR: A Simple and Effective Multimodal Retrieval and Answer Refinement Framework for Multimodal Question Answering](https://mlsanity.com/paper/2408.08521/) (2024)
- [Face-to-Face Contrastive Learning for Social Intelligence Question-Answering](https://mlsanity.com/paper/2208.01036/) (2022)
- [VEGAS: Towards Visually Explainable and Grounded Artificial Social Intelligence](https://mlsanity.com/paper/2504.02227/) (2025)
- [Extending Compositional Attention Networks for Social Reasoning in Videos](https://mlsanity.com/paper/2210.01191/) (2022)
- [Multi-Modal Video Topic Segmentation with Dual-Contrastive Domain Adaptation](https://mlsanity.com/paper/2312.00220/) (2023)

## Abstract (arXiv)

We present MCQA, a learning-based algorithm for multimodal question answering. MCQA explicitly fuses and aligns the multimodal input (i.e. text, audio, and video), which forms the context for the query (question and answer). Our approach fuses and aligns the question and the answer within this context. Moreover, we use the notion of co-attention to perform cross-modal alignment and multimodal context-query alignment. Our context-query alignment module matches the relevant parts of the multimodal context and the query with each other and aligns them to improve the overall performance. We evaluate the performance of MCQA on Social-IQ, a benchmark dataset for multimodal question answering. We compare the performance of our algorithm with prior methods and observe an accuracy improvement of 4-7%.