# Leveraging Audio-LLMs to Filter Speech-to-Speech Training Data

arXiv: https://arxiv.org/abs/2606.13507 · HTML: https://mlsanity.com/paper/2606.13507/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 28 references that are in the map.
- First posted on arXiv 11 Jun 2026.

## Builds on (most cited first, 23 of 23)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities](https://mlsanity.com/paper/2402.01831/) (2024)
- [Direct speech-to-speech translation with discrete units](https://mlsanity.com/paper/2107.05604/) (2021)
- [Sequence-to-Sequence Models Can Directly Translate Foreign Speech](https://mlsanity.com/paper/1703.08581/) (2017)
- [Direct speech-to-speech translation with a sequence-to-sequence model](https://mlsanity.com/paper/1904.06037/) (2019)
- [CVSS Corpus and Massively Multilingual Speech-to-Speech Translation](https://mlsanity.com/paper/2201.03713/) (2022)
- [Curriculum Learning for Domain Adaptation in Neural Machine Translation](https://mlsanity.com/paper/1905.05816/) (2019)
- [LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages](https://mlsanity.com/paper/2407.05975/) (2024)
- [Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection](https://mlsanity.com/paper/1809.00068/) (2018)
- [Effective Parallel Corpus Mining using Bilingual Sentence Embeddings](https://mlsanity.com/paper/1807.11906/) (2018)
- [SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations](https://mlsanity.com/paper/2211.04508/) (2022)
- [TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation](https://mlsanity.com/paper/2205.12523/) (2022)
- [Low-Resource Corpus Filtering using Multilingual Sentence Embeddings](https://mlsanity.com/paper/1906.08885/) (2019)
- [Pseudo Label Is Better Than Human Label](https://mlsanity.com/paper/2203.12668/) (2022)
- [Audio Large Language Models Can Be Descriptive Speech Quality Evaluators](https://mlsanity.com/paper/2501.17202/) (2025)
- [Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and C50 room acoustics estimation](https://mlsanity.com/paper/2210.13248/) (2022)
- [A Case Study on Filtering for End-to-End Speech Translation](https://mlsanity.com/paper/2402.01945/) (2024)

## Abstract (arXiv)

Large-scale mined corpora provide abundant training data for end-to-end speech-to-speech translation (S2ST) but may contain noise, misalignment, and semantic errors. Filtering noisy data is crucial to maintain robust speech translation performance. We study how to train an audio-language model to make keep/drop decisions on paired speech directly from audio. To obtain reliable supervision without manual labels, we adopt a scalable two-stage Rank-to-Distill strategy. A lightweight ranker generates keep/drop pseudo-labels from noisy speech pairs, then trains an audio large language model to predict keep/drop directly from raw paired speech. The resulting model jointly captures acoustic fidelity and cross-lingual semantic consistency for the selection of speech-conditioned data. Experiments on CVSS-C and SpeechMatrix show consistent improvements over unfiltered training, yielding up to +1.4 ASR-BLEU for end-to-end S2ST.