mlsanityData to 14 Oct 2026

EmphAssess : a Prosodic Benchmark on Assessing Emphasis Transfer in Speech-to-Speech Models

arXiv 2312.14069 ↗First posted 21 Dec 2023cs.CL cs.SD eess.AScited by 22

Not on the map. Why?

Abstract (arXiv)

We introduce EmphAssess, a prosodic benchmark designed to evaluate the capability of speech-to-speech models to encode and reproduce prosodic emphasis. We apply this to two tasks: speech resynthesis and speech-to-speech translation. In both cases, the benchmark evaluates the ability of the model to encode emphasis in the speech input and accurately reproduce it in the output, potentially across a change of speaker and language. As part of the evaluation pipeline, we introduce EmphaClass, a new model that classifies emphasis at the frame or word level.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 23 of its 48 references that are in the map. First posted on arXiv 21 Dec 2023.

Builds on

23 of 48 references.

References · 23

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  5. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  6. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  7. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  8. 2021Generative Spoken Language Modeling from Raw Audiocited by 340
  9. 2023WhisperX: Time-Accurate Speech Transcription of Long-Form Audiocited by 326
  10. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
  11. 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 156
  12. 2021Textless Speech-to-Speech Translation on Real Datacited by 121
11 more
  1. 2021Text-Free Prosody-Aware Generative Spoken Language Modelingcited by 119
  2. 2019Direct speech-to-speech translation with a sequence-to-sequence modelcited by 118
  3. 2023EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesiscited by 98
  4. 2021Translatotron 2: High-quality direct speech-to-speech translation with voice preservationcited by 71
  5. 2022On the Utility of Self-supervised Models for Prosody-related Taskscited by 42
  6. 2022Probing phoneme, language and speaker information in unsupervised speech representationscited by 33
  7. 2023ProsAudit, a prosodic benchmark for self-supervised speech modelscited by 25
  8. 2020Prosodic Prominence and Boundaries in Sequence-to-Sequence Speech Synthesiscited by 10
  9. 2022IQDUBBING: Prosody modeling based on discrete self-supervised speech representation for expressive voice conversioncited by 7
  10. 2021Deep Learning For Prominence Detection In Children's Read Speechcited by 6
  11. 2023Towards cross-language prosody transfer for dialogcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 10; 2026: 9.

22 papers.

Sort:

By area:

  1. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 39
  2. 2025AudioJudge: Understanding What Works in Large Audio Model Based Speech Evaluationcited by 21
  3. 2025Thinking with Sound: Audio Chain-of-Thought Enables Multimodal Reasoning in Large Audio-Language Modelscited by 9
  4. 2026Now You Hear Me: Audio Narrative Attacks Against Large Audio-Language Modelscited by 8
  5. 2025WHISTRESS: Enriching Transcriptions with Sentence Stress Detectioncited by 7
  6. 2025StressTest: Can YOUR Speech LM Handle the Stress?cited by 7
  7. 2024Can LLMs Understand the Implication of Emphasized Sentences in Dialogue?cited by 5
  8. 2024LightHouse: A Survey of AGI Hallucinationcited by 4
  9. 2025Segmentation-Variant Codebooks for Preservation of Paralinguistic and Prosodic Informationcited by 4
  10. 2025ProsodyLM: Uncovering the Emerging Prosody Processing Capabilities in Speech Language Modelscited by 4
  11. 2024Learning Fine-Grained Controllability on Speech Generation via Efficient Fine-Tuningcited by 1
  12. 2025Benchmarking Prosody Encoding in Discrete Speech Tokenscited by 1
10 more
  1. 2026Knowing What to Stress: A Discourse-Conditioned Text-to-Speech Benchmarkcited by 1
  2. 2026STEB: A Speech-to-Speech Translation Expressiveness Benchmark for Evaluating Beyond Translation Fidelitycited by 1
  3. 2025Emphasis Sensitivity in Speech Representationscited by 0
  4. 2025Centering Emotion Hotspots: Multimodal Local-Global Fusion and Cross-Modal Alignment for Emotion Recognition in Conversationscited by 0
  5. 2026Evaluating and Preserving Lexical Stress in English-to-Chinese Speech-to-Speech Translationcited by 0
  6. 2026ParaPairAudioBench: Paralinguistic Pairwise Audio Benchmark for LALM-as-a-Judgecited by 0
  7. 2026Do Speech Emphasis Models Generalize across Languages and Emotions?cited by 0
  8. 2026Unlocking Speech-Text Compositional Powers: Instruction-Following Speech Language Models without Instruction Tuningcited by 0
  9. 2026Iterative Self-Learning for Expressive Text-to-Speech Synthesiscited by 0
  10. 2026A Novel Sentence Stress Detection Framework Leveraging Auxiliary Word-Stress Modeling and Loss Optimizationcited by 0