mlsanityData to 7 Oct 2026

MSLM-S2ST: A Multitask Speech Language Model for Textless Speech-to-Speech Translation with Speaker Style Preservation

arXiv 2403.12408 ↗First posted 19 Mar 2024cs.CL cs.SD eess.AScited by 11

Not on the map. Why?

Abstract (arXiv)

There have been emerging research interest and advances in speech-to-speech translation (S2ST), translating utterances from one language to another. This work proposes Multitask Speech Language Model (MSLM), which is a decoder-only speech language model trained in a multitask setting. Without reliance on text training data, our model is able to support multilingual S2ST with speaker style preserved.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 21 of its 24 references that are in the map. First posted on arXiv 19 Mar 2024.

Builds on

21 of 24 references.

References · 21

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  6. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  7. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  8. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  9. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 333
  10. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  11. 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modelingcited by 166
  12. 2021Textless Speech-to-Speech Translation on Real Datacited by 121
9 more
  1. 2021Text-Free Prosody-Aware Generative Spoken Language Modelingcited by 119
  2. 2023Textually Pretrained Speech Language Modelscited by 99
  3. 2023Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation taskscited by 71
  4. 2021Translatotron 2: High-quality direct speech-to-speech translation with voice preservationcited by 69
  5. 2022SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translationscited by 35
  6. 2022SpeechLMScore: Evaluating speech generation using speech language modelcited by 19
  7. 2023PolyVoice: Language Models for Speech to Speech Translationcited by 19
  8. 2023Multilingual Speech-to-Speech Translation into Multiple Target Languagescited by 5
  9. 2023Speech-to-Speech Translation with Discrete-Unit-Based Style Transfercited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 5; 2026: 2.

11 papers.

  1. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 42
  2. 2025Slamming: Training a Speech Language Model on One GPU in a Daycited by 11
  3. 2024TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservationcited by 10
  4. 2024SeamlessExpressiveLM: Speech Language Model for Expressive Speech-to-Speech Translation with Chain-of-Thoughtcited by 9
  5. 2024LAST: Language Model Aware Speech Tokenizationcited by 9
  6. 2025UniSS: Unified Expressive Speech-to-Speech Translation with Your Voicecited by 6
  7. 2024Direct Speech-to-Speech Neural Machine Translation: A Surveycited by 5
  8. 2025Scheduled Interleaved Speech-Text Training for Speech-to-Speech Translation with LLMscited by 3
  9. 2026STEB: A Speech-to-Speech Translation Expressiveness Benchmark for Evaluating Beyond Translation Fidelitycited by 1
  10. 2025GmSLM : Generative Marmoset Spoken Language Modelingcited by 0
  11. 2026Kraken: LLM-based Speech-to-Speech Translation via Low-bitrate VQ and Dual-path Source Conditioningcited by 0