mlsanityData to 14 Oct 2026

Can LLMs Understand the Implication of Emphasized Sentences in Dialogue?

arXiv 2406.11065 ↗First posted 16 Jun 2024cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Emphasis is a crucial component in human communication, which indicates the speaker's intention and implication beyond pure text in dialogue. While Large Language Models (LLMs) have revolutionized natural language processing, their ability to understand emphasis in dialogue remains unclear. This paper introduces Emphasized-Talk, a benchmark with emphasis-annotated dialogue samples capturing the implications of emphasis. We evaluate various LLMs, both open-source and commercial, to measure their performance in understanding emphasis. Additionally, we propose an automatic evaluation pipeline using GPT-4, which achieves a high correlation with human rating. Our findings reveal that although commercial LLMs generally perform better, there is still significant room for improvement in comprehending emphasized sentences.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 14 of its 34 references that are in the map. First posted on arXiv 16 Jun 2024.

Builds on

14 of 34 references.

References · 14

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  4. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  5. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 486
  6. 2022DailyTalk: Spoken Dialogue Dataset for Conversational Text-to-Speechcited by 69
  7. 2024Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversationscited by 45
  8. 2023Paralinguistics-Enhanced Large Language Modeling of Spoken Dialoguecited by 25
  9. 2023EmphAssess : a Prosodic Benchmark on Assessing Emphasis Transfer in Speech-to-Speech Modelscited by 22
  10. 2019Predicting Prosodic Prominence from Text with Pre-trained Contextualized Word Representationscited by 14
  11. 2022BERT, can HE predict contrastive focus? Predicting and controlling prominence in neural TTS using a language modelcited by 10
  12. 2021Emphasis control for parallel neural TTScited by 9
2 more
  1. 2021Deep Learning For Prominence Detection In Children's Read Speechcited by 6
  2. 2023Crowdsourced and Automatic Speech Prominence Estimationcited by 6

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 4.

5 papers.

  1. 2025Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilitiescited by 100
  2. 2024Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedbackcited by 23
  3. 2025StressTest: Can YOUR Speech LM Handle the Stress?cited by 7
  4. 2025EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Spoken Dialogue Systemscited by 2
  5. 2025Mind the Quote: Enabling Quotation-Aware Dialogue in LLMs via Plug-and-Play Modulescited by 0