mlsanityData to 14 Oct 2026

MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning

arXiv 2304.08981 ↗First posted 18 Apr 2023cs.CL cs.CVcited by 33

Not on the map. Why?

Abstract (arXiv)

The first Multimodal Emotion Recognition Challenge (MER 2023) was successfully held at ACM Multimedia. The challenge focuses on system robustness and consists of three distinct tracks: (1) MER-MULTI, where participants are required to recognize both discrete and dimensional emotions; (2) MER-NOISE, in which noise is added to test videos for modality robustness evaluation; (3) MER-SEMI, which provides a large amount of unlabeled samples for semi-supervised learning. In this paper, we introduce the motivation behind this challenge, describe the benchmark dataset, and provide some statistics about participants. To continue using this dataset after MER 2023, please sign a new End User License Agreement and send it to our official email address [email protected]. We believe this high-quality dataset can become a new benchmark in multimodal emotion recognition, especially for the Chinese research community.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 37 of its 88 references that are in the map. First posted on arXiv 18 Apr 2023.

Builds on

37 of 88 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2017Squeeze-and-Excitation Networkscited by 4,789
  7. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  10. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  11. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  12. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
25 more
  1. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  2. 2016CNN Architectures for Large-Scale Audio Classificationcited by 704
  3. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
  4. 2016MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognitioncited by 624
  5. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 505
  6. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 425
  7. 2013Challenges in Representation Learning: A report on three machine learning contestscited by 302
  8. 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 266
  9. 2017Recent Trends in Deep Learning Based Natural Language Processingcited by 263
  10. 2018DialogueRNN: An Attentive RNN for Emotion Detection in Conversationscited by 218
  11. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  12. 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 125
  13. 2019AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognitioncited by 87
  14. 2022GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversationcited by 87
  15. 2022UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learningcited by 77
  16. 2020Deep Partial Multi-View Learningcited by 71
  17. 2016AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challengecited by 69
  18. 2021Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognitioncited by 68
  19. 2019Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysiscited by 39
  20. 2021The MuSe 2021 Multimodal Sentiment Analysis Challenge: Sentiment, Emotion, Physiological-Emotion, and Stresscited by 27
  21. 2015EmoNets: Multimodal deep learning approaches for emotion recognition in videocited by 20
  22. 2022Analyzing Modality Robustness in Multimodal Sentiment Analysiscited by 20
  23. 2022Efficient Multimodal Transformer with Dual-Level Feature Restoration for Robust Multimodal Sentiment Analysiscited by 20
  24. 2018EmotiW 2018: Audio-Video, Student Engagement and Group-Level Affect Predictioncited by 18
  25. 2019Unsupervised Representation Learning with Future Observation Prediction for Speech Emotion Recognitioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 12; 2025: 12; 2026: 8.

33 papers.

Sort:

By area:

  1. 2024Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuningcited by 121
  2. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 113
  3. 2025AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Modelscited by 44
  4. 2024EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmarkcited by 40
  5. 2023GPT-4V with Emotion: A Zero-shot Benchmark for Generalized Emotion Recognitioncited by 38
  6. 2025EmoBench-M: Benchmarking Emotional Intelligence for Multimodal Large Language Modelscited by 32
  7. 2024HiCMAE: Hierarchical Contrastive Masked Autoencoder for Self-Supervised Audio-Visual Emotion Recognitioncited by 30
  8. 2024MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognitioncited by 23
  9. 2024Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Surveycited by 16
  10. 2024Emotion and Intent Joint Understanding in Multimodal Conversation: A Benchmarking Datasetcited by 15
  11. 2025OSUM: Advancing Open Speech Understanding Models with Limited Resources in Academiacited by 13
  12. 2025Face-LLaVA: Facial Expression and Attribute Understanding through Instruction Tuningcited by 11
21 more
  1. 2026OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computingcited by 6
  2. 2024Leveraging Contrastive Learning and Self-Training for Multimodal Emotion Recognition with Limited Labeled Samplescited by 5
  3. 2026Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understandingcited by 5
  4. 2025EmoSign: A Multimodal Dataset for Understanding Emotions in American Sign Languagecited by 4
  5. 2025More Is Better: A MoE-Based Emotion Recognition Framework with Human Preference Alignmentcited by 4
  6. 2025StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient SpeechLLMscited by 4
  7. 2025RRPO: Robust Reward Policy Optimization for LLM-based Emotional TTScited by 4
  8. 2025Multimodal Video Emotion Recognition with Reliable Reasoning Priorscited by 3
  9. 2024Evaluation of data inconsistency for multi-modal sentiment analysiscited by 2
  10. 2024Enhancing Multimodal Sentiment Analysis for Missing Modality through Self-Distillation and Unified Modality Cross-Attentioncited by 2
  11. 2025UDDETTS: Unifying Discrete and Dimensional Emotions for Controllable Emotional Text-to-Speechcited by 2
  12. 2025A Unified Evaluation Framework for Multi-Annotator Tendency Learningcited by 2
  13. 2024Audio-Guided Fusion Techniques for Multimodal Emotion Analysiscited by 1
  14. 2024Early Joint Learning of Emotion Information Makes MultiModal Model Understand You Bettercited by 1
  15. 2026Nano-EmoX: Unifying Multimodal Emotional Intelligence from Perception to Empathycited by 1
  16. 2026Benchmarking Dynamic Affective Reasoning: A Viewer-Centric Video Emotion Datasetcited by 1
  17. 2024SZTU-CMU at MER2024: Improving Emotion-LLaMA with Conv-Attention for Multimodal Emotion Recognitioncited by 0
  18. 2026Understanding Affective Adaptation in Multimodal Foundation Models: Emergent Functional Specializationcited by 0
  19. 2026Variational Encoder--Multi-Decoder (VE-MD) for Privacy-by-functional-design (Group) Emotion Recognitioncited by 0
  20. 2026AffectSeek: Agentic Affective Understanding in Long Videos under Vague User Queriescited by 0
  21. 2026A Comprehensive Review of Multimodal Facial State Analysis: Tasks, Methods, and Resourcescited by 0