# Leveraging Acoustic Cues and Paralinguistic Embeddings to Detect Expression from Voice

arXiv: https://arxiv.org/abs/1907.00112 · HTML: https://mlsanity.com/paper/1907.00112/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2019.

## Built on by (most cited first, 7 of 7)

- [Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classification](https://mlsanity.com/paper/2102.07370/) (2021)
- [Investigating salient representations and label Variance in Dimensional Speech Emotion Analysis](https://mlsanity.com/paper/2312.16180/) (2023)
- [Speech Emotion: Investigating Model Representations, Multi-Task Learning and Knowledge Distillation](https://mlsanity.com/paper/2207.03334/) (2022)
- [Pre-trained Model Representations and their Robustness against Noise for Speech Emotion Analysis](https://mlsanity.com/paper/2303.03177/) (2023)
- [Modeling speech emotion with label variance and analyzing performance across speakers and unseen acoustic conditions](https://mlsanity.com/paper/2503.22711/) (2025)
- [Representing 'how you say' with 'what you say': English corpus of focused speech and text reflecting corresponding implications](https://mlsanity.com/paper/2203.15483/) (2022)
- [Exploring Teacher-Student Learning Approach for Multi-lingual Speech-to-Intent Classification](https://mlsanity.com/paper/2109.13486/) (2021)

## Abstract (arXiv)

Millions of people reach out to digital assistants such as Siri every day, asking for information, making phone calls, seeking assistance, and much more. The expectation is that such assistants should understand the intent of the users query. Detecting the intent of a query from a short, isolated utterance is a difficult task. Intent cannot always be obtained from speech-recognized transcriptions. A transcription driven approach can interpret what has been said but fails to acknowledge how it has been said, and as a consequence, may ignore the expression present in the voice. Our work investigates whether a system can reliably detect vocal expression in queries using acoustic and paralinguistic embedding. Results show that the proposed method offers a relative equal error rate (EER) decrease of 60% compared to a bag-of-word based system, corroborating that expression is significantly represented by vocal attributes, rather than being purely lexical. Addition of emotion embedding helped to reduce the EER by 30% relative to the acoustic embedding, demonstrating the relevance of emotion in expressive voice.