mlsanityData to 14 Oct 2026

PACT: A Pragmatic Prompting Framework for Expressive and Voice-Preserving Speech-to-Speech Translation

arXiv 2610.18247 ↗First posted 13 Oct 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Speech-to-speech translation should preserve not only what is said, but also who says it and how it is said. Existing end-to-end S2ST, speech MLLM, and cascade systems can preserve content or timbre, but often struggle to realize emotion, tone, rhythm, pauses, and pragmatic attitude in the target language. We propose PACT, a target-space pragmatic prompting framework for expressive and voice-preserving S2ST. PACT extracts source pragmatic cues, projects them into the target-language expression space, calibrates the projected representation to a realizable target subspace, and converts it into continuous prompts for a frozen speech generator. On bidirectional Spanish-English TOPI evaluation, PACT achieves the best overall performance among evaluated systems across content fidelity, voice preservation, and expression preservation metrics.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 32 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

25 of 32 references.

References · 25

  1. 2024Qwen2.5 Technical Reportcited by 4,942
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  5. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  6. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 334
  7. 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 205
  8. 2023Seamless: Multilingual Expressive and Streaming Speech Translationcited by 192
  9. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 179
  10. 2026Qwen3.5-Omni Technical Reportcited by 174
  11. 2021Direct speech-to-speech translation with discrete unitscited by 149
  12. 2021Translatotron 2: High-quality direct speech-to-speech translation with voice preservationcited by 71
13 more
  1. 2024Fish-Speech: Leveraging Large Language Models for Advanced Multilingual Text-to-Speech Synthesiscited by 48
  2. 2022Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentationcited by 44
  3. 2023OpenVoice: Versatile Instant Voice Cloningcited by 32
  4. 2023A Holistic Cascade System, benchmark, and Human Evaluation Protocol for Expressive Speech-to-Speech Translationcited by 11
  5. 2025UniSS: Unified Expressive Speech-to-Speech Translation with Your Voicecited by 7
  6. 2023Speech-to-Speech Translation with Discrete-Unit-Based Style Transfercited by 6
  7. 2024Direct Speech-to-Speech Neural Machine Translation: A Surveycited by 6
  8. 2022Dialogs Re-enacted Across Languagescited by 2
  9. 2023Expressive Machine Dubbing Through Phrase-level Cross-lingual Prosody Transfercited by 2
  10. 2024CrossVoice: Crosslingual Prosody Preserving Cascade-S2ST using Transfer Learningcited by 2
  11. 2022Representing 'how you say' with 'what you say': English corpus of focused speech and text reflecting corresponding implicationscited by 1
  12. 2025A Unit-based System and Dataset for Expressive Direct Speech-to-Speech Translationcited by 1
  13. 2025Dub-S2ST: Textless Speech-to-Speech Translation for Seamless Dubbingcited by 1

Built on by

Not cited yet.