LyS at SemEval-2024 Task 3: An Early Prototype for End-to-End Multimodal Emotion Linking as Graph-Based Parsing
Not on the map. Why?
Abstract (arXiv)
This paper describes our participation in SemEval 2024 Task 3, which focused on Multimodal Emotion Cause Analysis in Conversations. We developed an early prototype for an end-to-end system that uses graph-based methods from dependency parsing to identify causal emotion relations in multi-party conversations. Our model comprises a neural transformer-based encoder for contextualizing multimodal conversation data and a graph-based decoder for generating the adjacency matrix scores of the causal graph. We ranked 7th out of 15 valid and official submissions for Subtask 1, using textual inputs only. We also discuss our participation in Subtask 2 during post-evaluation using multi-modal inputs.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 23 of its 38 references that are in the map. First posted on arXiv 10 May 2024.
Builds on
23 of 38 references.
References · 23
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2020Longformer: The Long-Document Transformercited by 3,413
- 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
- 2021ViViT: A Video Vision Transformercited by 1,620
- 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
11 more
- 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 482
- 2016Deep Biaffine Attention for Neural Dependency Parsingcited by 334
- 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 321
- 2013Challenges in Representation Learning: A report on three machine learning contestscited by 302
- 2019Emotion-Cause Pair Extraction: A New Task to Emotion Analysis in Textscited by 83
- 2021Facial Expression Recognition with Visual Transformers and Attentional Selective Fusioncited by 49
- 2018Simpler but More Accurate Semantic Dependency Parsingcited by 46
- 2021SpanEmo: Casting Multi-label Emotion Classification as Span-predictioncited by 37
- 2021Multimodal Emotion-Cause Pair Extraction in Conversationscited by 31
- 2023Conversation Understanding using Relational Temporal Graph Neural Networks with Auxiliary Cross-Modality Interactioncited by 18
- 2023HCAM -- Hierarchical Cross Attention Model for Multi-modal Emotion Recognitioncited by 11
Built on by
Citing papers per year: 2024: 1.
1 paper.