mlsanityData to 14 Oct 2026

Cross-Task Multi-Branch Vision Transformer for Facial Expression and Mask Wearing Classification

arXiv 2404.14606 ↗First posted 22 Apr 2024cs.CV cs.AIcited by 16

Not on the map. Why?

Abstract (arXiv)

With wearing masks becoming a new cultural norm, facial expression recognition (FER) while taking masks into account has become a significant challenge. In this paper, we propose a unified multi-branch vision transformer for facial expression recognition and mask wearing classification tasks. Our approach extracts shared features for both tasks using a dual-branch architecture that obtains multi-scale feature representations. Furthermore, we propose a cross-task fusion phase that processes tokens for each task with separate branches, while exchanging information using a cross attention module. Our proposed framework reduces the overall complexity compared with using separate networks for both tasks by the simple yet effective cross-task fusion phase. Extensive experiments demonstrate that our proposed model performs better than or on par with different state-of-the-art methods on both facial expression recognition and facial mask wearing classification task.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 28 of its 57 references that are in the map. First posted on arXiv 22 Apr 2024.

Builds on

28 of 57 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  5. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  6. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 919
  7. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  8. 2013Challenges in Representation Learning: A report on three machine learning contestscited by 302
  9. 2019Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognitioncited by 127
  10. 2023NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Spacecited by 90
  11. 2023Deception Detection from Linguistic and Physiological Data Streams Using Bimodal Convolutional Neural Networkscited by 56
  12. 2024Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Reviewcited by 53
16 more
  1. 2021FaceX-Zoo: A PyTorch Toolbox for Face Recognitioncited by 49
  2. 2024Exploring Diverse Methods in Visual Question Answeringcited by 45
  3. 2018Contextual Hourglass Network for Semantic Segmentation of High Resolution Aerial Imagerycited by 43
  4. 2023MmAP : Multi-modal Alignment Prompt for Cross-domain Multi-task Learningcited by 41
  5. 2023Bridging the Domain Gap: Self-Supervised 3D Scene Understanding with Foundation Modelscited by 27
  6. 2021Multimodal Semi-Supervised Learning for 3D Objectscited by 24
  7. 2024Particle Filter SLAM for Vehicle Localizationcited by 23
  8. 2024News Recommendation with Attention Mechanismcited by 15
  9. 2024Rumor Detection with a novel graph neural network approachcited by 15
  10. 2022Class-Level Confidence Based 3D Semi-Supervised Learningcited by 14
  11. 2018Vehicle Pose and Shape Estimation through Multiple Monocular Visioncited by 13
  12. 2023Building Lane-Level Maps from Aerial Imagescited by 13
  13. 2024Image Captioning in news report scenariocited by 12
  14. 2024The application of Augmented Reality (AR) in Remote Work and Educationcited by 8
  15. 2024Deep Reinforcement Learning for Mobile Robot Path Planningcited by 5
  16. 2024Research on emotionally intelligent dialogue generation based on automatic dialogue systemcited by 5

Built on by

Cited by year

Citing papers per year: 2024: 15; 2025: 1.

16 papers.

Sort:

By area:

  1. 2024Harnessing XGBoost for Robust Biomarker Selection of Obsessive-Compulsive Disorder (OCD) from Adolescent Brain Cognitive Development (ABCD) datacited by 26
  2. 2024RAG-RLRC-LaySum at BioLaySumm: Integrating Retrieval-Augmented Generation and Readability Control for Layman Summarization of Biomedical Textscited by 23
  3. 2024Enhance Image-to-Image Generation with LLaVA-generated Promptscited by 20
  4. 2024Image anomaly detection and prediction scheme based on SSA optimized ResNet50-BiGRU modelcited by 18
  5. 2024Efficiency optimization of large-scale language models based on deep learning in natural language processing taskscited by 14
  6. 2024Exploration of Attention Mechanism-Enhanced Deep Learning Models in the Mining of Medical Textual Datacited by 11
  7. 2024Exploration of Multi-Scale Image Fusion Systems in Intelligent Medical Image Analysiscited by 10
  8. 2024Multi-Modal UAV Detection, Classification and Tracking Algorithm -- Technical Report for CVPR 2024 UG2 Challengecited by 9
  9. 2024Advanced AI Framework for Enhanced Detection and Assessment of Abdominal Trauma: Integrating 3D Segmentation with 2D CNN and RNN Modelscited by 9
  10. 2024Applying Conditional Generative Adversarial Networks for Imaging Diagnosiscited by 9
  11. 2024Recording Brain Activity While Listening to Music Using Wearable EEG Devices Combined with Bidirectional Long Short-Term Memory Networkscited by 8
  12. 2024Advancements in Feature Extraction Recognition of Medical Imaging Systems Through Deep Learning Techniquecited by 7
4 more
  1. 2024Affective Behaviour Analysis via Progressive Learningcited by 7
  2. 2025OpenFace 3.0: A Lightweight Multitask System for Comprehensive Facial Behavior Analysiscited by 5
  3. 2024Advanced Multimodal Deep Learning Architecture for Image-Text Matchingcited by 4
  4. 2024Improvement in Facial Emotion Recognition using Synthetic Data Generated by Diffusion Modelcited by 0