# GigaAM: Efficient Self-Supervised Learner for Speech Recognition

arXiv: https://arxiv.org/abs/2506.01192 · HTML: https://mlsanity.com/paper/2506.01192/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2025.

## Built on by (most cited first, 3 of 3)

- [Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoders](https://mlsanity.com/paper/2606.07473/) (2026)
- [GigaAM Multilingual: Foundation Model for Underrepresented Languages](https://mlsanity.com/paper/2607.10371/) (2026)
- [GigaChat Audio: Time-aware Large Audio Language Model](https://mlsanity.com/paper/2607.10387/) (2026)

## Abstract (arXiv)

Self-Supervised Learning (SSL) has demonstrated strong performance in speech processing, particularly in automatic speech recognition. In this paper, we explore an SSL pretraining framework that leverages masked language modeling with targets derived from a speech recognition model. We also present chunkwise attention with dynamic chunk size sampling during pretraining to enable both full-context and streaming fine-tuning. Our experiments examine scaling with respect to model size and the amount of data. Using our method, we train the GigaAM family of models, including a state-of-the-art model for Russian speech recognition that outperforms Whisper-large-v3 by 50%. We have released our foundation and ASR models, along with the inference code, under the MIT license as open-source resources to the research community. Available at https://github.com/salute-developers/gigaam.