mlsanityData to 14 Oct 2026

Instituto de Telecomunicacc~oes at IWSLT 2025: Aligning Small-Scale Speech and Language Models for Speech-to-Text Learning

arXiv 2506.17019 ↗First posted 20 Jun 2025cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper presents the IT-IST submission to the IWSLT 2025 Shared Task on Instruction Following Speech Processing. We submit results for the Short Track, i.e., speech recognition, translation, and spoken question answering. Our model is a unified speech-to-text model that integrates a pre-trained continuous speech encoder and text decoder through a first phase of modality alignment and a second phase of instruction fine-tuning. Crucially, we focus on using small-scale language model backbones (< 2B) and restrict to high-quality, CC-BY data along with synthetic data generation to supplement existing resources.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 32 references that are in the map. First posted on arXiv 20 Jun 2025.

Builds on

22 of 32 references.

References · 22

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  4. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  5. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  6. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  7. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  8. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 235
  9. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 207
  10. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
  11. 2024Small Language Models: Survey, Measurements, and Insightscited by 112
  12. 2024EuroLLM: Multilingual Language Models for Europecited by 106
10 more
  1. 2021The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usagecited by 101
  2. 2023VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translationcited by 87
  3. 2018Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehensioncited by 74
  4. 2023Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation taskscited by 72
  5. 2023Speech Translation with Large Language Models: An Industrial Practicecited by 28
  6. 2024Twists, Humps, and Pebbles: Multilingual Speech Recognition Models Exhibit Gender Performance Gapscited by 25
  7. 2024Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?cited by 23
  8. 2024Synthetic Query Generation using Large Language Models for Virtual Assistantscited by 9
  9. 2025From TOWER to SPIRE: Adding the Speech Modality to a Translation-Specialist LLMcited by 8
  10. 2024Towards Probing Speech-Specific Risks in Large Multimodal Models: A Taxonomy, Benchmark, and Insightscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data
  2. 2024DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignment
  3. 2023Harnessing the Zero-Shot Power of Instruction-Tuned Large Language Model in End-to-End Speech Recognition
  4. 2026Alignment Is All You Need: Instruction-Free Training for General Audio-Language Models
  5. 2025AzeroS: Extending LLM to Speech with Self-Generated Instruction-Free Tuning
  6. 2025From Alignment to Advancement: Bootstrapping Audio-Language Alignment with Synthetic Data
  7. 2025Understanding the Modality Gap: An Empirical Study on the Speech-Text Alignment Mechanism of Large Speech Language Models
  8. 2023BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writing