mlsanityData to 7 Oct 2026

Multilingual Long-Form Speech Instruction Following: KIT's Submission to IWSLT 2026

arXiv 2606.04730 ↗First posted 3 Jun 2026cs.CL eess.AScited by 1

Not on the map. Why?

Abstract (arXiv)

With the advent of Large Language Models, single-task and token-based multi-task models have evolved into instruction-based systems that infer task and target language implicitly from natural language prompts. This trend is reflected in IWSLT's Instruction Following Track, which this year introduced new tasks including an unknown surprise task, posing a genuine challenge against overfitting to known tasks. We present KIT's submission to the Long and Short Instruction Following tracks in the unconstrained setting. Our approach combines a general data augmentation pipeline that converts short-form corpora into long-form training data through segment concatenation, LLM-based label generation, and cross-lingual translation, yielding over 1M instances across six tasks and four languages. We further show that likelihood-based re-ranking, while highly effective for ASR, systematically degrades semantic tasks by spuriously selecting candidates generated from segmented audio processing rather than holistic long-form inference, a failure mode resolved by combining likelihood with Minimum Bayes Risk decoding.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

22 of 30 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  4. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 230
  5. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
  6. 2023SeamlessM4T: Massively Multilingual & Multimodal Machine Translationcited by 124
  7. 2019Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debatescited by 114
  8. 2025MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmarkcited by 101
  9. 2020CoVoST 2 and Massively Multilingual Speech-to-Text Translationcited by 73
  10. 2025Exploring the Impact of Temperature on Large Language Models:Hot or Cold?cited by 40
  11. 2022SHAS: Approaching optimal Segmentation for End-to-End Speech Translationcited by 26
  12. 2024From Text Segmentation to Smart Chaptering: A Novel Benchmark for Structuring Video Transcriptionscited by 16
10 more
  1. 2025MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talkscited by 14
  2. 2023End-to-End Evaluation for Low-Latency Simultaneous Speech Translationcited by 10
  3. 2024Contrastive Learning for Task-Independent SpeechLLM-Pretrainingcited by 8
  4. 2025Weight Factorization and Centralization for Continual Learning in Speech Recognitioncited by 6
  5. 2025Summarizing Speech: A Comprehensive Surveycited by 5
  6. 2025KIT's Offline Speech Translation and Instruction Following Submission for IWSLT 2025cited by 4
  7. 2025Do Slides Help? Multi-modal Context for Automatic Transcription of Conference Talkscited by 4
  8. 2025NUTSHELL: A Dataset for Abstract Generation from Scientific Talkscited by 3
  9. 2026Beyond Transcripts: A Renewed Perspective on Audio Chapteringcited by 3
  10. 2025BOOM: Beyond Only One Modality KIT's Multimodal Multilingual Lecture Companioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.