mlsanityData to 14 Oct 2026

StressTransfer: Stress-Aware Speech-to-Speech Translation with Emphasis Preservation

arXiv 2510.13194 ↗First posted 15 Oct 2025cs.CL cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

We propose a stress-aware speech-to-speech translation (S2ST) system that preserves word-level emphasis by leveraging LLMs for cross-lingual emphasis conversion. Our method translates source-language stress into target-language tags that guide a controllable TTS model. To overcome data scarcity, we developed a pipeline to automatically generate aligned training data and introduce the "LLM-as-Judge" for evaluation. Experiments show our approach substantially outperforms baselines in preserving emphasis while maintaining comparable translation quality, speaker intent, and naturalness. Our work highlights the importance of prosody in translation and provides an effective, data-efficient solution for preserving paralinguistic cues in S2ST.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 15 Oct 2025.

Builds on

22 of 29 references.

References · 22

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  3. 2024Qwen2 Technical Reportcited by 2,120
  4. 2024Qwen2-Audio Technical Reportcited by 560
  5. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  6. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  7. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  8. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
  9. 2023Seamless: Multilingual Expressive and Streaming Speech Translationcited by 192
  10. 2021Direct speech-to-speech translation with discrete unitscited by 149
  11. 2023On decoder-only architecture for speech-to-text and large language model integrationcited by 145
  12. 2023SeamlessM4T: Massively Multilingual & Multimodal Machine Translationcited by 125
10 more
  1. 2019Direct speech-to-speech translation with a sequence-to-sequence modelcited by 118
  2. 2021Translatotron 2: High-quality direct speech-to-speech translation with voice preservationcited by 71
  3. 2022UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Unitscited by 60
  4. 2023Speech Translation with Large Language Models: An Industrial Practicecited by 28
  5. 2017Neural Lattice-to-Sequence Models for Uncertain Inputscited by 20
  6. 2024LLaST: Improved End-to-end Speech Translation System Leveraged by Large Language Modelscited by 12
  7. 2024TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservationcited by 10
  8. 2024SeamlessExpressiveLM: Speech Language Model for Expressive Speech-to-Speech Translation with Chain-of-Thoughtcited by 9
  9. 2025WHISTRESS: Enriching Transcriptions with Sentence Stress Detectioncited by 7
  10. 2025StressTest: Can YOUR Speech LM Handle the Stress?cited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.