# CLARE: Cognitive Load Assessment in REaltime with Multimodal Data

arXiv: https://arxiv.org/abs/2404.17098 · HTML: https://mlsanity.com/paper/2404.17098/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 65 references that are in the map.
- First posted on arXiv 26 Apr 2024.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Transformer Architecture for Stress Detection from ECG](https://mlsanity.com/paper/2108.09737/) (2021)
- [A dataset of continuous affect annotations and physiological signals for emotion analysis](https://mlsanity.com/paper/1812.02782/) (2018)
- [Transformer-based Self-supervised Multimodal Representation Learning for Wearable Emotion Recognition](https://mlsanity.com/paper/2303.17611/) (2023)
- [Multimodal Brain-Computer Interface for In-Vehicle Driver Cognitive Load Measurement: Dataset and Baselines](https://mlsanity.com/paper/2304.04273/) (2023)
- [Attentive Cross-modal Connections for Deep Multimodal Wearable-based Emotion Recognition](https://mlsanity.com/paper/2108.02241/) (2021)
- [AttX: Attentive Cross-Connections for Fusion of Wearable Signals in Emotion Recognition](https://mlsanity.com/paper/2206.04625/) (2022)
- [EEG-based Cognitive Load Classification using Feature Masked Autoencoding and Emotion Transfer Learning](https://mlsanity.com/paper/2308.00246/) (2023)
- [AVCAffe: A Large Scale Audio-Visual Dataset of Cognitive Load and Affect for Remote Work](https://mlsanity.com/paper/2205.06887/) (2022)

## Built on by (most cited first, 8 of 8)

- [SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses](https://mlsanity.com/paper/2510.15476/) (2025)
- [Streaming, Fast and Slow: Cognitive Load-Aware Streaming for Efficient LLM Serving](https://mlsanity.com/paper/2504.17999/) (2025)
- [One-Block Transformer (1BT) for EEG-Based Cognitive Workload Assessment](https://mlsanity.com/paper/2605.00856/) (2026)
- [GazeMind: A Gaze-Guided LLM Agent for Personalized Cognitive Load Assessment](https://mlsanity.com/paper/2605.05790/) (2026)
- [Multi-Domain EEG Representation Learning with Orthogonal Mapping and Attention-based Fusion for Cognitive Load Classification](https://mlsanity.com/paper/2511.12394/) (2025)
- [Graph-Based Learning of Spectro-Topographical EEG Representations with Gradient Alignment for Brain-Computer Interfaces](https://mlsanity.com/paper/2512.07820/) (2025)
- [Understanding Mental States in Active and Autonomous Driving with EEG](https://mlsanity.com/paper/2512.09190/) (2025)
- [CogAdapt: Adapting Clinical ECG Foundation Models for Wearable Cognitive Load Assessment](https://mlsanity.com/paper/2605.22774/) (2026)

## Abstract (arXiv)

We present a novel multimodal dataset for Cognitive Load Assessment in REal-time (CLARE). The dataset contains physiological and gaze data from 24 participants with self-reported cognitive load scores as ground-truth labels. The dataset consists of four modalities, namely, Electrocardiography (ECG), Electrodermal Activity (EDA), Electroencephalogram (EEG), and Gaze tracking. To map diverse levels of mental load on participants during experiments, each participant completed four nine-minutes sessions on a computer-based operator performance and mental workload task (the MATB-II software) with varying levels of complexity in one minute segments. During the experiment, participants reported their cognitive load every 10 seconds. For the dataset, we also provide benchmark binary classification results with machine learning and deep learning models on two different evaluation schemes, namely, 10-fold and leave-one-subject-out (LOSO) cross-validation. Benchmark results show that for 10-fold evaluation, the convolutional neural network (CNN) based deep learning model achieves the best classification performance with ECG, EDA, and Gaze. In contrast, for LOSO, the best performance is achieved by the deep learning model with ECG, EDA, and EEG.