mlsanityData to 7 Oct 2026

A Multimodal Fusion Network For Student Emotion Recognition Based on Transformer and Tensor Product

arXiv 2403.08511 ↗First posted 13 Mar 2024cs.CVcited by 22

Not on the map. Why?

Abstract (arXiv)

This paper introduces a new multi-modal model based on the Transformer architecture and tensor product fusion strategy, combining BERT's text vectors and ViT's image vectors to classify students' psychological conditions, with an accuracy of 93.65%. The purpose of the study is to accurately analyze the mental health status of students from various data sources. This paper discusses modal fusion methods, including early, late and intermediate fusion, to overcome the challenges of integrating multi-modal information. Ablation studies compare the performance of different models and fusion techniques, showing that the proposed model outperforms existing methods such as CLIP and ViLBERT in terms of accuracy and inference speed. Conclusions indicate that while this model has significant advantages in emotion recognition, its potential to incorporate other data modalities provides areas for future research.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 7 of its 13 references that are in the map. First posted on arXiv 13 Mar 2024.

Builds on

7 of 13 references.

References · 7

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  5. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  6. 2019Multimodal Intelligence: Representation Learning, Information Fusion, and Applicationscited by 89
  7. 2023An Integrative Paradigm for Enhanced Stroke Prediction: Synergizing XGBoost and xDeepFM Algorithmscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 13; 2026: 1.

22 papers.

Sort:

By area:

  1. 2024Optimizing Automated Picking Systems in Warehouse Robots Using Machine Learningcited by 19
  2. 2024A Neural Matrix Decomposition Recommender System Model based on the Multimodal Large Language Modelcited by 12
  3. 2024Transformer-Based Classification Outcome Prediction for Multimodal Stroke Treatmentcited by 11
  4. 2025Research on the Design of a Short Video Recommendation System Based on Multimodal Information and Differential Privacycited by 9
  5. 2025User Behavior Analysis in Privacy Protection with Large Language Models: A Study on Privacy Preferences with Limited Datacited by 8
  6. 2025Research on Personalized Financial Product Recommendation by Integrating Large Language Models and Graph Neural Networkscited by 6
  7. 2025Research on Model Parallelism and Data Parallelism Optimization Methods in Large Language Model-Based Recommendation Systemscited by 6
  8. 2026Missing-by-Design: Certifiable Modality Deletion for Revocable Multimodal Sentiment Analysiscited by 6
  9. 2025Meta-Learning for Cold-Start Personalization in Prompt-Tuned LLMscited by 5
  10. 2025RLHF Fine-Tuning of LLMs for Alignment with Implicit User Feedback in Conversational Recommenderscited by 5
  11. 2024Analysis and Design of a Personalized Recommendation System Based on a Dynamic User Interest Modelcited by 4
  12. 2024Real-time Video Target Tracking Algorithm Utilizing Convolutional Neural Networks (CNN)cited by 4
10 more
  1. 2025Deep Learning Model Acceleration and Optimization Strategies for Real-Time Recommendation Systemscited by 3
  2. 2024Detecting and Classifying Defective Products in Images Using YOLOcited by 2
  3. 2025LLM-Augmented Symptom Analysis for Cardiovascular Disease Risk Prediction: A Clinical NLPcited by 2
  4. 2025Efficient Cold-Start Recommendation via BPE Token-Level Embedding Initialization with LLMcited by 2
  5. 2024Multi-modal clothing recommendation model based on large model and VAE enhancementcited by 1
  6. 2024Optimization and Scalability of Collaborative Filtering Algorithms in Large Language Modelscited by 1
  7. 2025The Transparent Earth: A Multimodal Foundation Model for the Earth's Subsurfacecited by 1
  8. 2025Instructional Prompt Optimization for Few-Shot LLM-Based Recommendations on Cold-Start Userscited by 1
  9. 2025Using Vision Language Models to Detect Students' Academic Emotion through Facial Expressionscited by 0
  10. 2025Multimodal Foundation Model-Driven User Interest Modeling and Behavior Analysis on Short Video Platformscited by 0