# AudioInsight: Detecting Social Contexts Relevant to Social Anxiety from Speech

arXiv: https://arxiv.org/abs/2407.14458 · HTML: https://mlsanity.com/paper/2407.14458/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 26 references that are in the map.
- First posted on arXiv 19 Jul 2024.

## Builds on (most cited first, 5 of 5)

- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [From Personalized Medicine to Population Health: A Survey of mHealth Sensing Techniques](https://mlsanity.com/paper/2107.00948/) (2021)
- [Finding Dory in the Crowd: Detecting Social Interactions using Multi-Modal Mobile Sensing](https://mlsanity.com/paper/1809.00947/) (2018)
- [Wearable Sensor-based Multimodal Physiological Responses of Socially Anxious Individuals across Social Contexts](https://mlsanity.com/paper/2304.01293/) (2023)

## Built on by (most cited first, 2 of 2)

- [CALLM: Understanding Cancer Survivors' Emotions and Intervention Opportunities via Mobile Diaries and Context-Aware Language Models](https://mlsanity.com/paper/2503.10707/) (2025)
- [Affect Models Have Weak Generalizability to Atypical Speech](https://mlsanity.com/paper/2504.16283/) (2025)

## Abstract (arXiv)

During social interactions, understanding the intricacies of the context can be vital, particularly for socially anxious individuals. While previous research has found that the presence of a social interaction can be detected from ambient audio, the nuances within social contexts, which influence how anxiety provoking interactions are, remain largely unexplored. As an alternative to traditional, burdensome methods like self-report, this study presents a novel approach that harnesses ambient audio segments to detect social threat contexts. We focus on two key dimensions: number of interaction partners (dyadic vs. group) and degree of evaluative threat (explicitly evaluative vs. not explicitly evaluative). Building on data from a Zoom-based social interaction study (N=52 college students, of whom the majority N=45 are socially anxious), we employ deep learning methods to achieve strong detection performance. Under sample-wide 5-fold Cross Validation (CV), our model distinguished dyadic from group interactions with 90% accuracy and detected evaluative threat at 83%. Using a leave-one-group-out CV, accuracies were 82% and 77%, respectively. While our data are based on virtual interactions due to pandemic constraints, our method has the potential to extend to diverse real-world settings. This research underscores the potential of passive sensing and AI to differentiate intricate social contexts, and may ultimately advance the ability of context-aware digital interventions to offer personalized mental health support.