mlsanityData to 14 Oct 2026

MemeCap: A Dataset for Captioning and Interpreting Memes

arXiv 2305.13703 ↗First posted 23 May 2023cs.CLcited by 46

Not on the map. Why?

Abstract (arXiv)

Memes are a widely popular tool for web users to express their thoughts using visual metaphors. Understanding memes requires recognizing and interpreting visual metaphors with respect to the text inside or around the meme, often while employing background knowledge and reasoning abilities. We present the task of meme captioning and release a new dataset, MemeCap. Our dataset contains 6.3K memes along with the title of the post containing the meme, the meme captions, the literal image caption, and the visual metaphors. Despite the recent success of vision and language (VL) models on tasks such as image captioning and visual question answering, our extensive experiments using state-of-the-art VL models show that they still struggle with visual metaphors, and perform substantially worse than humans.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). Builds on 25 of its 44 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

25 of 44 references.

References · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  10. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  11. 2023LIMA: Less Is More for Alignmentcited by 1,100
  12. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
13 more
  1. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 738
  2. 2023Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Textcited by 208
  3. 2022Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contestcited by 98
  4. 2023Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Imagescited by 80
  5. 2021MERMAID: Metaphor Generation with Symbolism and Discriminative Decodingcited by 49
  6. 2021MelBERT: Metaphor Detection via Contextualized Late Interaction using Metaphorical Identification Theoriescited by 46
  7. 2021It's not Rocket Science : Interpreting Figurative Language in Narrativescited by 39
  8. 2022Metaphors in Pre-Trained Language Models: Probing and Generalization Across Datasets and Languagescited by 30
  9. 2021Figurative Language in Recognizing Textual Entailmentcited by 26
  10. 2020Clue: Cross-modal Coherence Modeling for Caption Generationcited by 24
  11. 2023IRFL: Image Recognition of Figurative Languagecited by 23
  12. 2023Characterizing the Entities in Harmful Memes: Who is the Hero, the Villain, the Victim?cited by 14
  13. 2022DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformationcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 15; 2025: 11; 2026: 14.

46 papers.

Sort:

By area:

  1. 2023Valley: Video Assistant with Large Language model Enhanced abilitYcited by 234
  2. 2023Video Understanding with Large Language Models: A Surveycited by 206
  3. 2023Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generationcited by 56
  4. 2023I Spy a Metaphor: Large Language Models and Diffusion Models Co-Create Visual Metaphorscited by 31
  5. 2024HEMM: Holistic Evaluation of Multimodal Foundation Modelscited by 21
  6. 2024ConTextual: Evaluating Context-Sensitive Text-Rich Visual Reasoning in Large Multimodal Modelscited by 19
  7. 2024Understanding Figurative Meaning through Explainable Visual Entailmentcited by 19
  8. 2024Modularized Networks for Few-shot Hateful Meme Detectioncited by 15
  9. 2024Innovative Thinking, Infinite Humor: Humor Research of Large Language Models through Structured Thought Leapscited by 13
  10. 2025Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark Studycited by 12
  11. 2024Cracking the Code of Juxtaposition: Can AI Models Understand the Humorous Contradictionscited by 11
  12. 2023A Template Is All You Memecited by 9
34 more
  1. 2024YesBut: A High-Quality Annotated Multimodal Dataset for evaluating Satire Comprehension capability of Vision-Language Modelscited by 9
  2. 2024AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language Modelscited by 8
  3. 2024Unveiling the Invisible: Captioning Videos with Metaphorscited by 7
  4. 2024Can We Predict Performance of Large Models across Vision-Language Tasks?cited by 6
  5. 2025BottleHumor: Self-Informed Humor Explanation using the Information Bottleneck Principlecited by 6
  6. 2023Social Meme-ing: Measuring Linguistic Variation in Memescited by 5
  7. 2024VAGUE: Visual Contexts Clarify Ambiguous Expressionscited by 5
  8. 2025Who's Laughing Now? An Overview of Computational Humour Generation and Explanationcited by 5
  9. 2024SAFE-MEME: Structured Reasoning Framework for Robust Hate Speech Detection in Memescited by 4
  10. 2026PEST: Parameter Efficient Steering of Blackbox VLMs via Agentic Few-shot Alignment for Hateful Meme Moderationcited by 4
  11. 2024RoMemes: A multimodal meme corpus for the Romanian languagecited by 3
  12. 2025Concept Drift Guided LayerNorm Tuning for Efficient Multimodal Metaphor Identificationcited by 3
  13. 2025What Media Frames Reveal About Stance: A Dataset and Study about Memes in Climate Change Discoursecited by 3
  14. 2026On the Wings of Imagination: Conflicting Script-based Multi-role Framework for Humor Caption Generationcited by 3
  15. 2025Large Vision-Language Models for Knowledge-Grounded Data Annotation of Memescited by 2
  16. 2025MINT: Multimodal Instruction Tuning with Multimodal Interaction Groupingcited by 2
  17. 2024Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysiscited by 1
  18. 2025Oogiri-Master: Benchmarking Humor Understanding via Oogiricited by 1
  19. 2025From Perception to Punchline: Empowering VLM with the Art of In-the-wild Memecited by 1
  20. 2026MER-Bench: A Comprehensive Benchmark for Multimodal Meme Reappraisalcited by 1
  21. 2024BCAmirs at SemEval-2024 Task 4: Beyond Words: A Multimodal and Multilingual Exploration of Persuasion in Memescited by 0
  22. 2025Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Modelscited by 0
  23. 2025What is Beneath Misogyny: Misogynous Memes Classification and Explanationcited by 0
  24. 2026Memes-as-Replies: Can Models Select Humorous Manga Panel Responses?cited by 0
  25. 2026Culture-Aware Humorous Captioning: Multimodal Humor Generation across Cultural Contextscited by 0
  26. 2026I Know What You Meme, Even If it Emerged Today: Understanding Evolving Memes through Open-World Knowledge Acquisitioncited by 0
  27. 2026Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challengescited by 0
  28. 2026Towards High-Level Semantic Intelligencecited by 0
  29. 2026MoCA: Implicit Social Context Analysiscited by 0
  30. 2026TransMeme: A Multi-Agent Framework for Cross-Cultural Meme Transcreationcited by 0
  31. 2026MemeBridge: A Dataset for Benchmarking and Mitigating the Bidirectional Cultural Gap in Meme Interpretationcited by 0
  32. 2026MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Modelscited by 0
  33. 2026Learn Before You Judge: Progressive Knowledge-to-Decision Alignment for Explainable Hateful Meme Detectioncited by 0
  34. 2026PRISM: A Category-Theoretic Framework for Measuring and Refining Multimodal Analogiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MemeLens: Multilingual Multitask VLMs for Memes
  2. 2025MemeReaCon: Probing Contextual Meme Understanding in Large Vision-Language Models
  3. 2026I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memes
  4. 2024XMeCap: Meme Caption Generation with Sub-Image Adaptability
  5. 2025On VLMs for Diverse Tasks in Multimodal Meme Classification
  6. 2026MemeBench: What LVLMs Miss When Interpreting Culture-Dependent Memes
  7. 2025Beyond Meme Templates: Limitations of Visual Similarity Measures in Meme Matching
  8. 2026Beyond a Joke: Multi-Angle Reasoning for Detecting and Explaining Harmful Humor in Memes