Learning Audio Representations with MLPs
Not on the map. Why?
Abstract (arXiv)
In this paper, we propose an efficient MLP-based approach for learning audio representations, namely timestamp and scene-level audio embeddings. We use an encoder consisting of sequentially stacked gated MLP blocks, which accept 2D MFCCs as inputs. In addition, we also provide a simple temporal interpolation-based algorithm for computing scene-level embeddings from timestamp embeddings. The audio representations generated by our method are evaluated across a diverse set of benchmarks at the Holistic Evaluation of Audio Representations (HEAR) challenge, hosted at the NeurIPS 2021 competition track. We achieved first place on the Speech Commands (full), Speech Commands (5 hours), and the Mridingham Tonic benchmarks. Furthermore, our approach is also the most resource-efficient among all the submitted methods, in terms of both the number of model parameters and the time required to compute embeddings.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 55 references that are in the map. First posted on arXiv 16 Mar 2022.
Builds on
33 of 55 references.
References · 33
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
- 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
- 2021Transformers in Vision: A Surveycited by 1,009
- 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
21 more
- 2021AST: Audio Spectrogram Transformercited by 634
- 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 563
- 2021Pay Attention to MLPscited by 367
- 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 339
- 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
- 2016SoundNet: Learning Sound Representations from Unlabeled Videocited by 244
- 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 186
- 2021SSAST: Self-Supervised Audio Spectrogram Transformercited by 166
- 2020VoxLingua107: a Dataset for Spoken Language Recognitioncited by 118
- 2018CREPE: A Convolutional Representation for Pitch Estimationcited by 109
- 2021Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognitioncited by 68
- 2019Deep Contextualized Acoustic Representations For Semi-Supervised Speech Recognitioncited by 66
- 2022Audio Self-supervised Learning: A Surveycited by 65
- 2021Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNetcited by 61
- 2021Keyword Transformer: A Self-Attention Model for Keyword Spottingcited by 59
- 2021Multi-Format Contrastive Learning of Audio Representationscited by 39
- 2021Learning Efficient Representations for Keyword Spotting with Triplet Losscited by 24
- 2021One Billion Audio Sounds from GPU-enabled Modular Synthesiscited by 11
- 2018Audio-based identification of beehive statescited by 7
- 2021Stride and Translation Invariance in CNNscited by 7
- 2021Attention-Free Keyword Spottingcited by 2
Built on by
Not cited yet.