mlsanityData to 7 Oct 2026

AVCAffe: A Large Scale Audio-Visual Dataset of Cognitive Load and Affect for Remote Work

arXiv 2205.06887 ↗First posted 13 May 2022cs.HC cs.CV cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

We introduce AVCAffe, the first Audio-Visual dataset consisting of Cognitive load and Affect attributes. We record AVCAffe by simulating remote work scenarios over a video-conferencing platform, where subjects collaborate to complete a number of cognitively engaging tasks. AVCAffe is the largest originally collected (not collected from the Internet) affective dataset in English language. We recruit 106 participants from 18 different countries of origin, spanning an age range of 18 to 57 years old, with a balanced male-female ratio. AVCAffe comprises a total of 108 hours of video, equivalent to more than 58,000 clips along with task-based self-reported ground truth labels for arousal, valence, and cognitive load attributes such as mental demand, temporal demand, effort, and a few others. We believe AVCAffe would be a challenging benchmark for the deep learning research community given the inherent difficulty of classifying affect and cognitive load in particular. Moreover, our dataset fills an existing timely gap by facilitating the creation of learning systems for better self-management of remote work meetings, and further study of hypotheses regarding the impact of remote work on cognitive load and affective states.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 25 of its 67 references that are in the map. First posted on arXiv 13 May 2022.

Builds on

25 of 67 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  7. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  8. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  9. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  10. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  11. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  12. 2019Self-Supervised Learning by Cross-Modal Audio-Video Clusteringcited by 264
13 more
  1. 2019Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentationcited by 258
  2. 2020Audio-Visual Instance Discrimination with Cross-Modal Agreementcited by 192
  3. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  4. 2021BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representationcited by 106
  5. 2020Self-supervised ECG Representation Learning for Emotion Recognitioncited by 99
  6. 2018Multi-attention Recurrent Network for Human Communication Comprehensioncited by 89
  7. 2019SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wildcited by 62
  8. 2019Exploiting multi-CNN features in CNN-RNN based Dimensional Emotion Recognition on the OMG in-the-wild Datasetcited by 39
  9. 2019Self-supervised Learning for ECG-based Emotion Recognitioncited by 32
  10. 2020K-EmoCon, a multimodal sensor dataset for continuous emotion recognition in naturalistic conversationscited by 30
  11. 2018A Multi-Task Learning & Generation Framework: Valence-Arousal, Action Units & Primary Expressionscited by 25
  12. 2020Semi-Supervised Object Detection with Sparsely Annotated Datasetcited by 10
  13. 2019Classification of Cognitive Load and Expertise for Adaptive Simulation using Deep Multitask Learningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 0; 2026: 3.

5 papers.

  1. 2023SVFAP: Self-supervised Video Facial Affect Perceivercited by 18
  2. 2024CLARE: Cognitive Load Assessment in REaltime with Multimodal Datacited by 8
  3. 2026Predicting Cognitive Load from Speech and Interaction Dynamics in Dyadic Conversationscited by 0
  4. 2026Reweighting Framewise Attention in Video Transformers for Facial Expression Understandingcited by 0
  5. 2026How Reliable Are Multimodal Signals of Conversational State? Evidence from Remote Dyadic Collaborative Taskscited by 0