mlsanityData to 7 Oct 2026

Comment-aided Video-Language Alignment via Contrastive Pre-training for Short-form Video Humor Detection

arXiv 2402.09055 ↗First posted 14 Feb 2024cs.CV cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

The growing importance of multi-modal humor detection within affective computing correlates with the expanding influence of short-form video sharing on social media platforms. In this paper, we propose a novel two-branch hierarchical model for short-form video humor detection (SVHD), named Comment-aided Video-Language Alignment (CVLA) via data-augmented multi-modal contrastive pre-training. Notably, our CVLA not only operates on raw signals across various modal channels but also yields an appropriate multi-modal representation by aligning the video and language components within a consistent semantic space. The experimental results on two humor detection datasets, including DY11k and UR-FUNNY, demonstrate that CVLA dramatically outperforms state-of-the-art and several competitive baseline approaches. Our dataset, code and model release at https://github.com/yliu-cs/CVLA.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 31 of its 52 references that are in the map. First posted on arXiv 14 Feb 2024.

Builds on

31 of 52 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  7. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  8. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  9. 2018SlowFast Networks for Video Recognitioncited by 1,862
  10. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  11. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  12. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
19 more
  1. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  2. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  3. 2020HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-trainingcited by 362
  4. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  5. 2020ActBERT: Learning Global-Local Video-Text Representationscited by 267
  6. 2022All in One: Exploring Unified Video-Language Pre-trainingcited by 171
  7. 2019UR-FUNNY: A Multimodal Language Dataset for Understanding Humorcited by 130
  8. 2020AVLnet: Learning Audio-Visual Language Representations from Instructional Videoscited by 110
  9. 2021VLM: Task-agnostic Video-Language Model Pre-training for Video Understandingcited by 102
  10. 2022Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learningcited by 59
  11. 2019Humor Detection: A Transformer Gets the Last Laughcited by 46
  12. 2021Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysiscited by 45
  13. 2022CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimationcited by 39
  14. 2021Understanding Chinese Video and Language via Contrastive Multimodal Pre-Trainingcited by 23
  15. 2022TVLT: Textless Vision-Language Transformercited by 22
  16. 2022Is Cross-Attention Preferable to Self-Attention for Multi-Modal Emotion Recognition?cited by 16
  17. 2020Uncertainty and Surprisal Jointly Deliver the Punchline: Exploiting Incongruity-Based Features for Humor Recognitioncited by 13
  18. 2021M2H2: A Multimodal Multiparty Hindi Dataset For Humor Recognition in Conversationscited by 7
  19. 2022Combining Humor and Sarcasm for Improving Political Parody Detectioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024Cracking the Code of Juxtaposition: Can AI Models Understand the Humorous Contradictionscited by 11
  2. 2024PiTe: Pixel-Temporal Alignment for Large Video-Language Modelcited by 7
  3. 2025When 'YES' Meets 'BUT': Can Large Models Comprehend Contradictory Humor Through Comparative Reasoning?cited by 5
  4. 2026CaRGo-T: Causal Reasoning Graph-of-Thought improves Multimodal Humor Comprehensioncited by 0