# Advancing Automated Spatio-Semantic Analysis in Picture Description Using Language Models

arXiv: https://arxiv.org/abs/2510.05128 · HTML: https://mlsanity.com/paper/2510.05128/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 21 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Builds on (most cited first, 5 of 5)

- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [To BERT or Not To BERT: Comparing Speech and Language-based Approaches for Alzheimer's Disease Detection](https://mlsanity.com/paper/2008.01551/) (2020)
- [Learning to Rank for Plausible Plausibility](https://mlsanity.com/paper/1906.02079/) (2019)
- [Early Dementia Detection Using Multiple Spontaneous Speech Prompts: The PROCESS Challenge](https://mlsanity.com/paper/2412.15230/) (2024)
- [Automated Extraction of Spatio-Semantic Graphs for Identifying Cognitive Impairment](https://mlsanity.com/paper/2502.01685/) (2025)

## Built on by (most cited first, 1 of 1)

- [Do LLMs Reliably Identify Correct Information Units in Aphasic Discourse?](https://mlsanity.com/paper/2606.15696/) (2026)

## Abstract (arXiv)

Current methods for automated assessment of cognitive-linguistic impairment via picture description often neglect the visual narrative path - the sequence and locations of elements a speaker described in the picture. Analyses of spatio-semantic features capture this path using content information units (CIUs), but manual tagging or dictionary-based mapping is labor-intensive. This study proposes a BERT-based pipeline, fine tuned with binary cross-entropy and pairwise ranking loss, for automated CIU extraction and ordering from the Cookie Theft picture description. Evaluated by 5-fold cross-validation, it achieves 93% median precision, 96% median recall in CIU detection, and 24% sequence error rates. The proposed method extracts features that exhibit strong Pearson correlations with ground truth, surpassing the dictionary-based baseline in external validation. These features also perform comparably to those derived from manual annotations in evaluating group differences via ANCOVA. The pipeline is shown to effectively characterize visual narrative paths for cognitive impairment assessment, with the implementation and models open-sourced to public.