mlsanityData to 7 Oct 2026

Spatio-Temporal Fuzzy-oriented Multi-Modal Meta-Learning for Fine-grained Emotion Recognition

arXiv 2412.13541 ↗First posted 18 Dec 2024cs.CV cs.LG cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

Fine-grained emotion recognition (FER) plays a vital role in various fields, such as disease diagnosis, personalized recommendations, and multimedia mining. However, existing FER methods face three key challenges in real-world applications: (i) they rely on large amounts of continuously annotated data to ensure accuracy since emotions are complex and ambiguous in reality, which is costly and time-consuming; (ii) they cannot capture the temporal heterogeneity caused by changing emotion patterns, because they usually assume that the temporal correlation within sampling periods is the same; (iii) they do not consider the spatial heterogeneity of different FER scenarios, that is, the distribution of emotion information in different data may have bias or interference. To address these challenges, we propose a Spatio-Temporal Fuzzy-oriented Multi-modal Meta-learning framework (ST-F2M). Specifically, ST-F2M first divides the multi-modal videos into multiple views, and each view corresponds to one modality of one emotion. Multiple randomly selected views for the same emotion form a meta-training task. Next, ST-F2M uses an integrated module with spatial and temporal convolutions to encode the data of each task, reflecting the spatial and temporal heterogeneity. Then it adds fuzzy semantic information to each task based on generalized fuzzy rules, which helps handle the complexity and ambiguity of emotions. Finally, ST-F2M learns emotion-related general meta-knowledge through meta-recurrent neural networks to achieve fast and robust fine-grained emotion recognition. Extensive experiments show that ST-F2M outperforms various state-of-the-art methods in terms of accuracy and model efficiency. In addition, we construct ablation studies and further analysis to explore why ST-F2M performs well.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 112 references that are in the map. First posted on arXiv 18 Dec 2024.

Builds on

34 of 112 references.

References · 34

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  3. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  4. 2017Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecastingcited by 1,125
  5. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  6. 2016MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videoscited by 193
  7. 2016Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distributioncited by 158
  8. 2018Look at Boundary: A Boundary-Aware Face Alignment Algorithmcited by 137
  9. 2021Fine-Grained Image Analysis with Deep Learning: A Surveycited by 135
  10. 2019Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advancescited by 118
  11. 2020Distortion Agnostic Deep Watermarkingcited by 88
  12. 2022UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognitioncited by 61
22 more
  1. 2022MARLIN: Masked Autoencoder for facial video Representation LearnINgcited by 55
  2. 2023Deep Imbalanced Learning for Multimodal Emotion Recognition in Conversationscited by 47
  3. 2022COGMEN: COntextualized GNN based Multimodal Emotion recognitioNcited by 29
  4. 2021Rethinking Graph Neural Architecture Search from Message-passingcited by 26
  5. 2016FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognitioncited by 25
  6. 2021Pre-training strategies and datasets for facial representation learningcited by 25
  7. 2022Tailor Versatile Multi-modal Learning for Multi-label Emotion Recognitioncited by 22
  8. 2020MER-GCN: Micro Expression Recognition Based on Relation Modeling with Graph Convolutional Networkcited by 21
  9. 2022GraphCFC: A Directed Graph Based Cross-Modal Feature Complementation Approach for Multimodal Conversational Emotion Recognitioncited by 20
  10. 2021Subtype-aware Unsupervised Domain Adaptation for Medical Diagnosiscited by 18
  11. 2019Variational Metric Scaling for Metric-Based Meta-Learningcited by 17
  12. 2022SepTr: Separable Transformer for Audio Spectrogram Processingcited by 15
  13. 2023Cluster-Level Contrastive Learning for Emotion Recognition in Conversationscited by 14
  14. 2023Towards Task Sampler Learning for Meta-Learningcited by 11
  15. 2020PropagationNet: Propagate Points to Curve to Learn Structure Informationcited by 10
  16. 2023Hacking Task Confounder in Meta-Learningcited by 8
  17. 2015A Real Time Facial Expression Classification System Using Local Binary Patternscited by 7
  18. 2016Facial expression recognition based on local region specific features and support vector machinescited by 5
  19. 2020Learning to Augment Expressions for Few-shot Fine-grained Facial Expression Recognitioncited by 3
  20. 2023EMERSK -- Explainable Multimodal Emotion Recognition with Situational Knowledgecited by 3
  21. 2024Meta-Auxiliary Learning for Micro-Expression Recognitioncited by 3
  22. 2020Meta Transfer Learning for Emotion Recognitioncited by 2

Built on by

Not cited yet.