Cross-modal Affinity-aligned Multimodal Learning Analytics for Predicting Student Collaboration Satisfaction in Game-Based Learning
Not on the map. Why?
Abstract (arXiv)
Collaborative game-based learning environments offer rich opportunities for small-group knowledge construction, yet automatically predicting student collaboration satisfaction remains challenging. A critical barrier is modality degradation: in educational deployments, individual modalities such as eye gaze exhibit inconsistent informativeness across student cohorts, causing implicit attention-based fusion to produce brittle multimodal representations. We propose the Affinity-Aligned Multimodal Learning Analytics (AAMLA) framework, whose core contribution is the Cross-modal Affinity-guided Modality Alignment (CAMA) module, which explicitly models inter-modal relationships via affinity matrices and enforces cross-modal consistency through contrastive learning, enabling adaptive suppression of uninformative modalities without discarding them. AAMLA further applies modality-specific projection layers to map heterogeneous features, including facial action units, head pose, eye gaze, and interaction trace logs, into a unified semantic space prior to alignment. Experiments on 50 middle school students in the EcoJourneys collaborative learning environment demonstrate consistent improvements over unimodal baselines and prior cross-attention approaches under standard and modality degradation conditions, with SHAP and t-SNE analyses confirming that CAMA produces robust, interpretable cross-modal representations for student collaboration modeling.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 51 references that are in the map. First posted on arXiv 16 May 2026.
Builds on
14 of 51 references.
References · 14
- 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 561
- 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 245
- 2023Multi-modal Learning with Missing Modality via Shared-Specific Feature Modellingcited by 163
- 2020Intriguing Properties of Contrastive Lossescited by 130
- 2021SelfDoc: Self-Supervised Document Representation Learningcited by 117
- 2023MmAP : Multi-modal Alignment Prompt for Cross-domain Multi-task Learningcited by 41
- 2023Structure-CLIP: Towards Scene Graph Knowledge to Enhance Multi-modal Structured Representationscited by 37
- 2023Connecting Multi-modal Contrastive Representationscited by 36
- 2023Quality-Agnostic Deepfake Detection with Intra-model Collaborative Learningcited by 32
- 2021Multimodal Engagement Analysis from Facial Videos in the Classroomcited by 27
- 2021Looking into Your Speech: Learning Cross-modal Affinity for Audio-visual Speech Separationcited by 27
- 2023LAMM: Label Alignment for Multi-Modal Prompt Learningcited by 15
Built on by
Not cited yet.