mlsanityData to 14 Oct 2026

Culture-Aware Humorous Captioning: Multimodal Humor Generation across Cultural Contexts

arXiv 2604.18091 ↗First posted 20 Apr 2026cs.CL cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment

Abstract (arXiv)

Recent multimodal large language models have shown promising ability in generating humorous captions for images, yet they still lack stable control over explicit cultural context, making it difficult to jointly maintain image relevance, contextual appropriateness, and humor quality under a specified cultural background. To address this limitation, we introduce a new multimodal generation task, culture-aware humorous captioning, which requires a model to generate a humorous caption conditioned on both an input image and a target cultural context. Captions generated under different cultural contexts are not expected to share the same surface form, but should remain grounded in similar visual situations or humorous rationales.To support this task, we establish a six-dimensional evaluation framework covering image relevance, contextual fit, semantic richness, reasonableness, humor, and creativity. We further propose a staged alignment framework that first initializes the model with high-resource supervision under the Western cultural context, then performs multi-dimensional preference alignment via judge-based GRPO with a Degradation-aware Prototype Repulsion Constraint to mitigate reward hacking in open-ended generation, and finally adapts the model to the Eastern cultural context with a small amount of supervision. Experimental results show that our method achieves stronger overall performance under the proposed evaluation framework, with particularly large gains in contextual fit and a better balance between image relevance and humor under cultural constraints.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 64 of its 75 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

64 of 75 references.

References · 64

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2025Qwen3-VL Technical Reportcited by 2,793
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  6. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  7. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  8. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  9. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  10. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  11. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  12. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
38 more
  1. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
  2. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 372
  3. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 367
  4. 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 217
  5. 2024Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judgescited by 205
  6. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 167
  7. 2023Osprey: Pixel Understanding with Visual Instruction Tuningcited by 159
  8. 2024Investigating Cultural Alignment of Large Language Modelscited by 143
  9. 2024LLaVA-Critic: Learning to Evaluate Multimodal Modelscited by 134
  10. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scalecited by 127
  11. 2025Describe Anything: Detailed Localized Image and Video Captioningcited by 84
  12. 2025MM-RLHF: The Next Step Forward in Multimodal LLM Alignmentcited by 83
  13. 2023Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generationcited by 56
  14. 2024Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsensecited by 51
  15. 2024LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steeringcited by 51
  16. 2023MemeCap: A Dataset for Captioning and Interpreting Memescited by 46
  17. 2024FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Modelcited by 45
  18. 2025CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Eracited by 41
  19. 2025Judge Anything: MLLM as a Judge Across Any Modalitycited by 34
  20. 2024Benchmarking Large Vision-Language Models via Directed Scene Graph for Comprehensive Image Captioningcited by 31
  21. 2024Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignmentcited by 29
  22. 2025Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimizationcited by 29
  23. 2024Multi-modal Preference Alignment Remedies Degradation of Visual Instruction Tuning on Language Modelscited by 26
  24. 2024Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Artcited by 26
  25. 2025Break the Checkbox: Challenging Closed-Style Evaluations of Cultural Alignment in LLMscited by 26
  26. 2024Extrinsic Evaluation of Cultural Competence in Large Language Modelscited by 24
  27. 2024Understanding Figurative Meaning through Explainable Visual Entailmentcited by 19
  28. 2024Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioningcited by 19
  29. 2025Crowd Comparative Reasoning: Unlocking Comprehensive Evaluations for LLM-as-a-Judgecited by 19
  30. 2025Cultural Learning-Based Culture Adaptation of Language Modelscited by 16
  31. 2025SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learningcited by 16
  32. 2025CARE: Multilingual Human Preference Learning for Cultural Awarenesscited by 15
  33. 2024XMeCap: Meme Caption Generation with Sub-Image Adaptabilitycited by 14
  34. 2024ControlCap: Controllable Region-level Captioningcited by 13
  35. 2025RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language Modelscited by 13
  36. 2025DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culturecited by 13
  37. 2023Learning Combinatorial Prompts for Universal Controllable Image Captioningcited by 12
  38. 2025Bridging the Creativity Understanding Gap: Small-Scale Human Alignment Enables Expert-Level Humor Ranking in LLMscited by 12

50 most cited of 64 All 64 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges
  2. 2026MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models
  3. 2024The Power of Many: Multi-Agent Multimodal Models for Cultural Image Captioning
  4. 2024CIC: A Framework for Culturally-Aware Image Captioning
  5. 2026Beyond Translation: Cross-Cultural Meme Transcreation with Vision-Language Models
  6. 2026Learning to Think Like a Cartoon Captionist: Incongruity-Resolution Supervision for Multimodal Humor Understanding
  7. 2025From Perception to Punchline: Empowering VLM with the Art of In-the-wild Meme
  8. 2025HumorPlanSearch: Structured Planning and HuCoT for Contextual AI Humor