mlsanityData to 14 Oct 2026

MUTAN: Multimodal Tucker Fusion for Visual Question Answering

arXiv 1705.06676 ↗First posted 18 May 2017cs.CVcited by 137

Not on the map. Why?

Abstract (arXiv)

Bilinear models provide an appealing framework for mixing and merging information in Visual Question Answering (VQA) tasks. They help to learn high level associations between question meaning and visual concepts in the image, but they suffer from huge dimensionality issues. We introduce MUTAN, a multimodal tensor-based Tucker decomposition to efficiently parametrize bilinear interactions between visual and textual representations. Additionally to the Tucker framework, we design a low-rank matrix-based decomposition to explicitly constrain the interaction rank. With MUTAN, we control the complexity of the merging scheme while keeping nice interpretable fusion relations. We show how our MUTAN model generalizes some of the latest VQA architectures, providing state-of-the-art results.

Key facts. Cited by 137 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 35; 2022: 34; 2023: 31; 2024: 28; 2025: 7; 2026: 2.

137 papers.

Sort:

By area:

  1. 2021An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQAcited by 301
  2. 2021Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learningcited by 187
  3. 2022CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoningcited by 120
  4. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  5. 2021Causal Attention for Vision-Language Taskscited by 111
  6. 2021Cross-Modal Progressive Comprehension for Referring Segmentationcited by 102
  7. 2021Multi-Modal Answer Validation for Knowledge-Based VQAcited by 79
  8. 2022Multi-modal Sensor Fusion for Auto Driving Perception: A Surveycited by 76
  9. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 76
  10. 2021Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answeringcited by 71
  11. 2021Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-trainingcited by 67
  12. 2021Adversarial Text-to-Image Synthesis: A Reviewcited by 52
38 more
  1. 2023SurgicalGPT: End-to-End Language-Vision GPT for Visual Question Answering in Surgerycited by 48
  2. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  3. 2023HiCLIP: Contrastive Language-Image Pretraining with Hierarchy-aware Attentioncited by 44
  4. 2024GeminiFusion: Efficient Pixel-wise Multimodal Fusion for Vision Transformercited by 44
  5. 2023BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Modelscited by 41
  6. 2022From Easy to Hard: Learning Language-guided Curriculum for Visual Question Answering on Remote Sensing Datacited by 39
  7. 2022Gender and Racial Bias in Visual Question Answering Datasetscited by 39
  8. 2024Surgical-LVLM: Learning to Adapt Large Vision-Language Model for Grounded Visual Question Answering in Robotic Surgerycited by 39
  9. 2022SwapMix: Diagnosing and Regularizing the Over-Reliance on Visual Context in Visual Question Answeringcited by 38
  10. 2022MuKEA: Multimodal Knowledge Extraction and Accumulation for Knowledge-based Visual Question Answeringcited by 36
  11. 2024Classifier-guided Gradient Modulation for Enhanced Multimodal Learningcited by 35
  12. 2021Select, Substitute, Search: A New Benchmark for Knowledge-Augmented Visual Question Answeringcited by 33
  13. 2021MixMo: Mixing Multiple Inputs for Multiple Outputs via Deep Subnetworkscited by 32
  14. 2023Tensor Networks Meet Neural Networks: A Survey and Future Perspectivescited by 32
  15. 2022Learning to Answer Visual Questions from Web Videoscited by 30
  16. 2022A Review on Methods and Applications in Multimodal Deep Learningcited by 29
  17. 2021LocalTrans: A Multiscale Local Transformer Network for Cross-Resolution Homography Estimationcited by 28
  18. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Modelscited by 28
  19. 2021RTIC: Residual Learning for Text and Image Composition using Graph Convolutional Networkcited by 27
  20. 2023Multi-modal Machine Learning in Engineering Design: A Review and Future Directionscited by 27
  21. 2021Recent Advances and Trends in Multimodal Deep Learning: A Reviewcited by 26
  22. 2023Mobile Foundation Model as Firmwarecited by 25
  23. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 25
  24. 2021Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledgecited by 24
  25. 2023Robust Visual Question Answering: Datasets, Methods, and Future Challengescited by 23
  26. 2024MLIP: Enhancing Medical Visual Representation with Divergence Encoder and Knowledge-guided Contrastive Learningcited by 23
  27. 2024PitVQA: Image-grounded Text Embedding LLM for Visual Question Answering in Pituitary Surgerycited by 23
  28. 2022VQA-GNN: Reasoning with Multimodal Knowledge via Graph Neural Networks for Visual Question Answeringcited by 22
  29. 2024Generative Multi-modal Models are Good Class-Incremental Learnerscited by 22
  30. 2022Towards Lightweight Transformer via Group-wise Transformation for Vision-and-Language Taskscited by 21
  31. 2023HRVQA: A Visual Question Answering Benchmark for High-Resolution Aerial Imagescited by 21
  32. 2024LOVA3: Learning to Visual Question Answering, Asking and Assessmentcited by 21
  33. 2022Phrase-Based Affordance Detection via Cyclic Bilateral Interactioncited by 19
  34. 2022GEB+: A Benchmark for Generic Event Boundary Captioning, Grounding and Retrievalcited by 19
  35. 2022Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical Alignmentcited by 19
  36. 2024Enhancing Visual Question Answering through Question-Driven Image Captions as Promptscited by 18
  37. 2021Comprehensive Visual Question Answering on Point Clouds through Compositional Scene Manipulationcited by 17
  38. 2023CAT-ViL: Co-Attention Gated Vision-Language Embedding for Visual Question Localized-Answering in Robotic Surgerycited by 17

50 most cited of 137 All 137 (JSON)