mlsanityData to 14 Oct 2026

Supervised Multimodal Bitransformers for Classifying Images and Text

arXiv 1909.02950 ↗First posted 6 Sep 2019cs.CL cs.CV cs.LGcited by 130

Not on the map. Why?

Abstract (arXiv)

Self-supervised bidirectional transformer models such as BERT have led to dramatic improvements in a wide variety of textual classification tasks. The modern digital world is increasingly multimodal, however, and textual information is often accompanied by other modalities such as images. We introduce a supervised multimodal bitransformer model that fuses information from text and image encoders, and obtain state-of-the-art performance on various multimodal classification benchmark tasks, outperforming strong baselines, including on hard test sets specifically designed to measure multimodal performance.

Key facts. Cited by 130 papers in the map (data through 14 Oct 2026). First posted on arXiv 6 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 29; 2022: 17; 2023: 26; 2024: 28; 2025: 18; 2026: 12.

130 papers.

Sort:

By area:

  1. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 661
  2. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 256
  3. 2022Are Multimodal Transformers Robust to Missing Modality?cited by 186
  4. 2022A Survey of Vision-Language Pre-Trained Modelscited by 177
  5. 2022Trusted Multi-View Classification with Dynamic Evidential Fusioncited by 168
  6. 2021Pretrained Transformers as Universal Computation Enginescited by 162
  7. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 146
  8. 2024CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuningcited by 108
  9. 2021Trusted Multi-View Classificationcited by 94
  10. 2021MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targetscited by 86
  11. 2023Provable Dynamic Fusion for Low-Quality Multimodal Datacited by 82
  12. 2021Detecting Harmful Memes and Their Targetscited by 78
38 more
  1. 2023Prompting for Multimodal Hateful Meme Classificationcited by 69
  2. 2021Human-Adversarial Visual Question Answeringcited by 57
  3. 2022Hate-CLIPper: Multimodal Hateful Meme Classification based on Cross-modal Interaction of CLIP Featurescited by 56
  4. 2021SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Imagescited by 50
  5. 2021A Survey on Multimodal Disinformation Detectioncited by 47
  6. 2023Pro-Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detectioncited by 47
  7. 2021Detecting Propaganda Techniques in Memescited by 44
  8. 2021Multimodal Learning using Optimal Transport for Sarcasm and Humor Detectioncited by 37
  9. 2022MultiViz: Towards Visualizing and Understanding Multimodal Modelscited by 36
  10. 2023Cross-Modal Fine-Tuning: Align then Refinecited by 34
  11. 2023Efficient Multimodal Fusion via Interactive Promptingcited by 32
  12. 2022High-Modality Multimodal Transformer: Quantifying Modality & Interaction Heterogeneity for High-Modality Representation Learningcited by 31
  13. 2022HIE-SQL: History Information Enhanced Network for Context-Dependent Text-to-SQL Semantic Parsingcited by 26
  14. 2024Towards Low-Resource Harmful Meme Detection with LMM Agentscited by 24
  15. 2023Robust Visual Question Answering: Datasets, Methods, and Future Challengescited by 23
  16. 2021Cross-Modal Retrieval Augmentation for Multi-Modal Classificationcited by 20
  17. 2022DISARM: Detecting the Victims Targeted by Harmful Memescited by 20
  18. 2022VoLTA: Vision-Language Transformer with Weakly-Supervised Local-Feature Alignmentcited by 20
  19. 2023BanglaAbuseMeme: A Dataset for Bengali Abusive Meme Classificationcited by 18
  20. 2023Moderating New Waves of Online Hate with Chain-of-Thought Reasoning in Large Language Modelscited by 18
  21. 2024Towards Explainable Harmful Meme Detection through Multimodal Debate between Large Language Modelscited by 18
  22. 2024Predictive Dynamic Fusioncited by 17
  23. 2021CMA-CLIP: Cross-Modality Attention CLIP for Image-Text Classificationcited by 16
  24. 2023Beneath the Surface: Unveiling Harmful Memes with Multimodal Reasoning Distilled from Large Language Modelscited by 15
  25. 2025MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detectioncited by 15
  26. 2023Characterizing the Entities in Harmful Memes: Who is the Hero, the Villain, the Victim?cited by 14
  27. 2023UniS-MMC: Multimodal Classification via Unimodality-supervised Multimodal Contrastive Learningcited by 13
  28. 2023Detecting and Correcting Hate Speech in Multimodal Memes with Large Visual Language Modelcited by 12
  29. 2021Analyzing Online Political Advertisementscited by 11
  30. 2022CARETS: A Consistency And Robustness Evaluative Test Suite for VQAcited by 11
  31. 2022SHAPE: An Unified Approach to Evaluate the Contribution and Cooperation of Individual Modalitiescited by 11
  32. 2023MEMEX: Detecting Explanatory Evidence for Memes via Knowledge-Enriched Contextualizationcited by 11
  33. 2024GAMED: Knowledge Adaptive Multi-Experts Decoupling for Multimodal Fake News Detectioncited by 11
  34. 2021Multimodal Fusion Refiner Networkscited by 9
  35. 2021Detection of Illicit Drug Trafficking Events on Instagram: A Deep Multimodal Multilabel Learning Approachcited by 9
  36. 2024MemeCraft: Contextual and Stance-Driven Multimodal Meme Generationcited by 9
  37. 2024M3Hop-CoT: Misogynous Meme Identification with Multimodal Multi-hop Chain-of-Thoughtcited by 9
  38. 2025See-Saw Modality Balance: See Gradient, and Sew Impaired Vision-Language Balance to Mitigate Dominant Modality Biascited by 9

50 most cited of 130 All 130 (JSON)