# Predicting States of Understanding in Explanatory Interactions Using Cognitive Load-Related Linguistic Cues

arXiv: https://arxiv.org/abs/2603.20079 · HTML: https://mlsanity.com/paper/2603.20079/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 58 references that are in the map.
- First posted on arXiv 20 Mar 2026.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [Testing the Predictions of Surprisal Theory in 11 Languages](https://mlsanity.com/paper/2307.03667/) (2023)
- [Universal Dependency Parsing from Scratch](https://mlsanity.com/paper/1901.10457/) (2019)
- [Surprise! Uniform Information Density Isn't the Whole Story: Predicting Surprisal Contours in Long-form Discourse](https://mlsanity.com/paper/2410.16062/) (2024)
- [Controlling Surprisal in Music Generation via Information Content Curve Matching](https://mlsanity.com/paper/2408.06022/) (2024)
- [Revisiting the Phenomenon of Syntactic Complexity Convergence on German Dialogue Data](https://mlsanity.com/paper/2408.12177/) (2024)

## Built on by (most cited first, 1 of 1)

- [Gaze as Evidence for Common Grounding: A Cross-Corpus Analysis of MapTask and MUNDEX](https://mlsanity.com/paper/2609.18011/) (2026)

## Abstract (arXiv)

We investigate how verbal and nonverbal linguistic features, exhibited by speakers and listeners in dialogue, can contribute to predicting the listener's state of understanding in explanatory interactions on a moment-by-moment basis. Specifically, we examine three linguistic cues related to cognitive load and hypothesised to correlate with listener understanding: the information value (operationalised with surprisal) and syntactic complexity of the speaker's utterances, and the variation in the listener's interactive gaze behaviour. Based on statistical analyses of the MUNDEX corpus of face-to-face dialogic board game explanations, we find that individual cues vary with the listener's level of understanding. Listener states ('Understanding', 'Partial Understanding', 'Non-Understanding' and 'Misunderstanding') were self-annotated by the listeners using a retrospective video-recall method. The results of a subsequent classification experiment, involving two off-the-shelf classifiers and a fine-tuned German BERT-based multimodal classifier, demonstrate that prediction of these four states of understanding is generally possible and improves when the three linguistic cues are considered alongside textual features.