mlsanityData to 7 Oct 2026

Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memes

arXiv 2012.07788 ↗First posted 14 Dec 2020cs.AI cs.CL cs.CVcited by 50

Not on the map. Why?

Abstract (arXiv)

This work presents Vilio, an implementation of state-of-the-art visio-linguistic models and their application to the Hateful Memes Dataset. The implemented models have been fitted into a uniform code-base and altered to yield better performance. The goal of Vilio is to provide a user-friendly starting point for any visio-linguistic problem. An ensemble of 5 different V+L models implemented in Vilio achieves 2nd place in the Hateful Memes Challenge out of 3,300 participants. The code is available at https://github.com/Muennighoff/vilio.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 6; 2023: 16; 2024: 10; 2025: 6; 2026: 3.

50 papers.

Sort:

By area:

  1. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  2. 2023Scaling Data-Constrained Language Modelscited by 391
  3. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 291
  4. 2024Generative Representational Instruction Tuningcited by 227
  5. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 171
  6. 2021MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targetscited by 85
  7. 2021Detecting Harmful Memes and Their Targetscited by 77
  8. 2023Prompting for Multimodal Hateful Meme Classificationcited by 68
  9. 2022Hate-CLIPper: Multimodal Hateful Meme Classification based on Cross-modal Interaction of CLIP Featurescited by 55
  10. 2021Disentangling Hate in Online Memescited by 46
  11. 2023Pro-Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detectioncited by 45
  12. 2023Decoding the Underlying Meaning of Multimodal Hateful Memescited by 38
38 more
  1. 2021Detecting Harmful Content On Online Platforms: What Platforms Need Vs. Where Research Efforts Gocited by 32
  2. 2022On the Evolution of (Hateful) Memes by Means of Multimodal Contrastive Learningcited by 26
  3. 2024Towards Low-Resource Harmful Meme Detection with LMM Agentscited by 23
  4. 2023Mapping Memes to Words for Multimodal Hateful Meme Classificationcited by 19
  5. 2022Emotion Based Hate Speech Detection using Multimodal Learningcited by 18
  6. 2022What do you MEME? Generating Explanations for Visual Semantic Role Labelling in Memescited by 18
  7. 2024Towards Explainable Harmful Meme Detection through Multimodal Debate between Large Language Modelscited by 17
  8. 2024Modularized Networks for Few-shot Hateful Meme Detectioncited by 15
  9. 2021Caption Enriched Samples for Improving Hateful Memes Detectioncited by 14
  10. 2023Beneath the Surface: Unveiling Harmful Memes with Multimodal Reasoning Distilled from Large Language Modelscited by 14
  11. 2025MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detectioncited by 14
  12. 2024Toxic Memes: A Survey of Computational Perspectives on the Detection and Explanation of Meme Toxicitiescited by 13
  13. 2023Detecting and Correcting Hate Speech in Multimodal Memes with Large Visual Language Modelcited by 12
  14. 2024MemeMQA: Multimodal Question Answering for Memes via Rationale-Based Inferencingcited by 12
  15. 2023MEMEX: Detecting Explanatory Evidence for Memes via Knowledge-Enriched Contextualizationcited by 11
  16. 2023OxfordTVG-HIC: Can Machine Make Humorous Captions from Images?cited by 9
  17. 2023HateProof: Are Hateful Meme Detection Systems really Robust?cited by 7
  18. 2023MemeFier: Dual-stage Modality Fusion for Image Meme Classificationcited by 6
  19. 2024Modality Invariant Multimodal Learning to Handle Missing Modalities: A Single-Branch Approachcited by 4
  20. 2024SAFE-MEME: Structured Reasoning Framework for Robust Hate Speech Detection in Memescited by 4
  21. 2021An Interpretable Approach to Hateful Meme Detectioncited by 3
  22. 2021Towards A Multi-agent System for Online Hate Speech Detectioncited by 2
  23. 2023Causal Intersectionality and Dual Form of Gradient Descent for Multimodal Analysis: a Case Study on Hateful Memescited by 2
  24. 2025Detecting and Mitigating Hateful Content in Multimodal Memes with Vision-Language Modelscited by 2
  25. 2025Representation Decomposition for Learning Similarity and Contrastness Across Modalities for Affective Computingcited by 2
  26. 2021Hateful Memes Challenge: An Enhanced Multimodal Frameworkcited by 1
  27. 2022Feels Bad Man: Dissecting Automated Hateful Meme Detection Through the Lens of Facebook's Challengecited by 1
  28. 2023A Review of Vision-Language Models and their Performance on the Hateful Memes Challengecited by 1
  29. 2023Unimodal Intermediate Training for Multimodal Meme Sentiment Classificationcited by 1
  30. 2024Prompt-enhanced Network for Hateful Meme Classificationcited by 1
  31. 2026All Changes May Have Invariant Principles: Improving Ever-Shifting Harmful Meme Detection via Design Concept Reproductioncited by 1
  32. 2021Diagnosing the Impact of AI on Radiology in Chinacited by 0
  33. 2023MATK: The Meme Analytical Tool Kitcited by 0
  34. 2025What's Pulling the Strings? Evaluating Integrity and Attribution in AI Training and Inference through Concept Shiftcited by 0
  35. 2025Unpacking Hateful Memes: Presupposed Context and False Claimscited by 0
  36. 2025Fall into a Pit, Gain in a Wit: Cognitive-Guided Harmful Meme Detection via Misjudgment Risk Pattern Retrievalcited by 0
  37. 2026Unveiling Covert Toxicity in Multimodal Data via Toxicity Association Graphs: A Graph-Based Metric and Interpretable Detection Frameworkcited by 0
  38. 2026SyRHM: Symbolic-Language-Enhanced Reasoning with Associative Retrieval for Zero-shot Harmful Meme Detectioncited by 0