mlsanityData to 14 Oct 2026

FAMA: The First Large-Scale Open-Science Speech Foundation Model for English and Italian

arXiv 2505.22759 ↗First posted 28 May 2025cs.CL cs.AI cs.SDcited by 2

Not on the map. Why?

Abstract (arXiv)

The development of speech foundation models (SFMs) like Whisper and SeamlessM4T has significantly advanced the field of speech processing. However, their closed nature--with inaccessible training data and code--poses major reproducibility and fair evaluation challenges. While other domains have made substantial progress toward open science by developing fully transparent models trained on open-source (OS) code and data, similar efforts in speech remain limited. To fill this gap, we introduce FAMA, the first family of open science SFMs for English and Italian, trained on 150k+ hours of OS speech data. Moreover, we present a new dataset containing 16k hours of cleaned and pseudo-labeled speech for both languages. Results show that FAMA achieves competitive performance compared to existing SFMs while being up to 8 times faster. All artifacts, including code, datasets, and models, are released under OS-compliant licenses, promoting openness in speech technology research.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 34 of its 44 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

34 of 44 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  4. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  5. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  7. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  8. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  9. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  10. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  11. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 483
  12. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
22 more
  1. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  2. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  3. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 229
  4. 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 181
  5. 2020fairseq S2T: Fast Speech-to-Text Modeling with fairseqcited by 176
  6. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 125
  7. 2024Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Modelscited by 122
  8. 2024EuroLLM: Multilingual Language Models for Europecited by 106
  9. 2023LLM360: Towards Fully Transparent Open-Source LLMscited by 104
  10. 2024OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformercited by 74
  11. 2023Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Datacited by 42
  12. 2022SHAS: Approaching optimal Segmentation for End-to-End Speech Translationcited by 26
  13. 2022CTC Alignments Improve Autoregressive Translationcited by 25
  14. 2024Twists, Humps, and Pebbles: Multilingual Speech Recognition Models Exhibit Gender Performance Gapscited by 25
  15. 2024Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?cited by 23
  16. 2024The Model Openness Framework: Promoting Completeness and Openness for Reproducibility, Transparency, and Usability in Artificial Intelligencecited by 23
  17. 2022Efficient yet Competitive Speech Translation: FBK@IWSLT2022cited by 14
  18. 2023When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLPcited by 14
  19. 2024MOSEL: 950,000 Hours of Speech Data for Open-Source Speech Foundation Model Training on EU Languagescited by 14
  20. 2023Preventing Catastrophic Forgetting in Continual Learning of New Natural Language Taskscited by 10
  21. 2024OpenOmni: A Collaborative Open Source Tool for Building Future-Ready Multimodal Conversational Agentscited by 3
  22. 2024A Case Study on Filtering for End-to-End Speech Translationcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Cross-Attention is Half Explanation in Speech-to-Text Modelscited by 1
  2. 2025MLMA: Towards Multilingual ASR With Mamba-based Architecturescited by 1