mlsanityData to 14 Oct 2026

Streaming Translation and Transcription Through Speech-to-Text Causal Alignment

arXiv 2603.11578 ↗First posted 12 Mar 2026cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Simultaneous machine translation (SiMT) has traditionally relied on offline machine translation models coupled with human-engineered heuristics or learned policies. We propose Hikari, a policy-free, end-to-end model for simultaneous speech-to-text translation and streaming transcription. We also introduce Decoder Time Dilation, a mechanism that counteracts the overrepresentation of WAIT tokens in training. We present a supervised fine-tuning strategy that trains the model to recover from delays, significantly improving the quality-latency trade-off. Despite its modest size, Hikari delivers competitive translation quality at consistently low latency, comparing favorably with published IWSLT 2026 submissions up to 38x larger and with proprietary API systems across en-ja, en-de, and en-ru. We release our model weights and code to facilitate further research.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 37 of its 68 references that are in the map. First posted on arXiv 12 Mar 2026.

Builds on

37 of 68 references.

References · 37

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  4. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  5. 2022Autoregressive Image Generation using Residual Quantizationcited by 722
  6. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  7. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  8. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 265
  9. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 226
  10. 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 156
  11. 2019Monotonic Infinite Lookback Attention for Simultaneous Machine Translationcited by 104
  12. 2016Learning to Translate in Real-time with Neural Machine Translationcited by 97
25 more
  1. 2020SimulEval: An Evaluation Toolkit for Simultaneous Translationcited by 94
  2. 2019Monotonic Multihead Attentioncited by 93
  3. 2016Can neural machine translation do simultaneous translation?cited by 86
  4. 2017Online and Linear-Time Attention by Enforcing Monotonic Alignmentscited by 85
  5. 2017Monotonic Chunkwise Attentioncited by 83
  6. 2025MinMo: A Multimodal Large Language Model for Seamless Voice Interactioncited by 79
  7. 2021Earnings-21: A Practical Benchmark for ASR in the Wildcited by 62
  8. 2023Efficient Sequence Transduction by Jointly Predicting Tokens and Durationscited by 58
  9. 2020Low-Latency Sequence-to-Sequence Speech Recognition and Translation by Partial Hypothesis Selectioncited by 49
  10. 2023AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translationcited by 35
  11. 2024TransLLaMa: LLM-based Simultaneous Translation Systemcited by 34
  12. 2022Attention as a Guide for Simultaneous Speech Translationcited by 32
  13. 2022CUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022cited by 30
  14. 2025Streaming Sequence-to-Sequence Learning with Delayed Streams Modelingcited by 27
  15. 2022SHAS: Approaching optimal Segmentation for End-to-End Speech Translationcited by 26
  16. 2024StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selectioncited by 19
  17. 2023Learning Optimal Policy for Simultaneous Machine Translation via Binary Searchcited by 15
  18. 2025High-Fidelity Simultaneous Speech-To-Speech Translationcited by 15
  19. 2025InfiniSST: Simultaneous Translation of Unbounded Speech with Large Language Modelcited by 8
  20. 2025Simultaneous Translation with Offline Speech and LLM Models in CUNI Submission to IWSLT 2025cited by 8
  21. 2026Simultaneous Speech-to-Speech Translation Without Aligned Datacited by 7
  22. 2023Tagged End-to-End Simultaneous Speech Translation Training using Simultaneous Interpretation Datacited by 6
  23. 2025CMU's IWSLT 2025 Simultaneous Speech Translation Systemcited by 2
  24. 2025MLLP-VRAIN UPV system for the IWSLT 2025 Simultaneous Speech Translation Translation taskcited by 2
  25. 2025BeaverTalk: Oregon State University's IWSLT 2025 Simultaneous Speech Translation Systemcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026A Pocket Offline Model for Simultaneous Speech Translation as CUNI Submission to IWSLT 2026cited by 0