Revisiting Direct Speech-to-Text Translation with Speech LLMs: Better Scaling than CoT Prompting?
On the map: LLM reasoning → Audio-language model reasoning
Abstract (arXiv)
Recent work on Speech-to-Text Translation (S2TT) has focused on LLM-based models, introducing the increasingly adopted Chain-of-Thought (CoT) prompting, where the model is guided to first transcribe the speech and then translate it. CoT typically outperforms direct prompting primarily because it can exploit abundant Automatic Speech Recognition (ASR) and Text-to-Text Translation (T2TT) datasets to explicitly model its steps. In this paper, we systematically compare CoT and Direct prompting under increasing amounts of S2TT data. To this end, we pseudo-label an ASR corpus by translating its transcriptions into six European languages, and train LLM-based S2TT systems with both prompting strategies at different data scales. Our results show that Direct improves more consistently as the amount of data increases, suggesting that it may become a more effective approach as larger S2TT resources are created.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 19 of its 25 references that are in the map. First posted on arXiv 3 Oct 2025.
Builds on
19 of 25 references.
Closest ancestors · 1
Same direction, newest first
Other references · 18
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
- 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
- 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
- 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
- 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 458
- 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
- 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 333
- 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 260
- 2023On decoder-only architecture for speech-to-text and large language model integrationcited by 144
- 2019Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debatescited by 114
- 2023SONAR: Sentence-Level Multimodal and Language-Agnostic Representationscited by 105
6 more
- 2023Textually Pretrained Speech Language Modelscited by 99
- 2023GlotLID: Language Identification for Low-Resource Languagescited by 37
- 2023Speech Translation with Large Language Models: An Industrial Practicecited by 28
- 2024Making LLMs Better Many-to-Many Speech-to-Text Translators with Curriculum Learningcited by 20
- 2020Improving Cross-Lingual Transfer Learning for End-to-End Speech Recognition with Speech Translationcited by 14
- 2024LLaST: Improved End-to-end Speech Translation System Leveraged by Large Language Modelscited by 12
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Listening or Reading? Evaluating Speech Awareness in Chain-of-Thought Speech-to-Text Translation
- 2025Speech-to-Text Translation with Phoneme-Augmented CoT: Enhancing Cross-Lingual Transfer in Low-Resource Scenarios
- 2026PROST-LLM: Progressively Enhancing the Speech-to-Speech Translation Capability in LLMs
- 2024What do Large Language Models Need for Machine Translation Evaluation?
- 2025NL in the Middle: Code Translation with LLMs and Intermediate Representations
- 2024Refining Translations with LLMs: A Constraint-Aware Iterative Prompting Approach
- 2024Low-Resource Machine Translation through Retrieval-Augmented LLM Prompting: A Study on the Mambai Language
- 2024Investigating Decoder-only Large Language Models for Speech-to-text Translation