# Can LLMs Understand the Implication of Emphasized Sentences in Dialogue?

arXiv: https://arxiv.org/abs/2406.11065 · HTML: https://mlsanity.com/paper/2406.11065/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 16 Jun 2024.

## Builds on (most cited first, 14 of 14)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [DailyTalk: Spoken Dialogue Dataset for Conversational Text-to-Speech](https://mlsanity.com/paper/2207.01063/) (2022)
- [Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations](https://mlsanity.com/paper/2402.12786/) (2024)
- [Paralinguistics-Enhanced Large Language Modeling of Spoken Dialogue](https://mlsanity.com/paper/2312.15316/) (2023)
- [EmphAssess : a Prosodic Benchmark on Assessing Emphasis Transfer in Speech-to-Speech Models](https://mlsanity.com/paper/2312.14069/) (2023)
- [Predicting Prosodic Prominence from Text with Pre-trained Contextualized Word Representations](https://mlsanity.com/paper/1908.02262/) (2019)
- [BERT, can HE predict contrastive focus? Predicting and controlling prominence in neural TTS using a language model](https://mlsanity.com/paper/2207.01718/) (2022)
- [Emphasis control for parallel neural TTS](https://mlsanity.com/paper/2110.03012/) (2021)
- [Deep Learning For Prominence Detection In Children's Read Speech](https://mlsanity.com/paper/2110.14273/) (2021)
- [Crowdsourced and Automatic Speech Prominence Estimation](https://mlsanity.com/paper/2310.08464/) (2023)

## Built on by (most cited first, 5 of 5)

- [Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities](https://mlsanity.com/paper/2503.04721/) (2025)
- [Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback](https://mlsanity.com/paper/2411.01834/) (2024)
- [StressTest: Can YOUR Speech LM Handle the Stress?](https://mlsanity.com/paper/2505.22765/) (2025)
- [EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Spoken Dialogue Systems](https://mlsanity.com/paper/2508.17623/) (2025)
- [Mind the Quote: Enabling Quotation-Aware Dialogue in LLMs via Plug-and-Play Modules](https://mlsanity.com/paper/2505.24292/) (2025)

## Abstract (arXiv)

Emphasis is a crucial component in human communication, which indicates the speaker's intention and implication beyond pure text in dialogue. While Large Language Models (LLMs) have revolutionized natural language processing, their ability to understand emphasis in dialogue remains unclear. This paper introduces Emphasized-Talk, a benchmark with emphasis-annotated dialogue samples capturing the implications of emphasis. We evaluate various LLMs, both open-source and commercial, to measure their performance in understanding emphasis. Additionally, we propose an automatic evaluation pipeline using GPT-4, which achieves a high correlation with human rating. Our findings reveal that although commercial LLMs generally perform better, there is still significant room for improvement in comprehending emphasized sentences.