# Learning from Between-class Examples for Deep Sound Recognition

arXiv: https://arxiv.org/abs/1711.10282 · HTML: https://mlsanity.com/paper/1711.10282/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2017.

## Built on by (most cited first, 30 of 54)

- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [AudioCLIP: Extending CLIP to Image, Text and Audio](https://mlsanity.com/paper/2106.13043/) (2021)
- [SSAST: Self-Supervised Audio Spectrogram Transformer](https://mlsanity.com/paper/2110.09784/) (2021)
- [Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model](https://mlsanity.com/paper/2304.13731/) (2023)
- [Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization](https://mlsanity.com/paper/2404.09956/) (2024)
- [PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation](https://mlsanity.com/paper/2102.01243/) (2021)
- [CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Models](https://mlsanity.com/paper/2310.08753/) (2023)
- [AlignMixup: Improving Representations By Interpolating Aligned Features](https://mlsanity.com/paper/2103.15375/) (2021)
- [MixMo: Mixing Multiple Inputs for Multiple Outputs via Deep Subnetworks](https://mlsanity.com/paper/2103.06132/) (2021)
- [A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainability](https://mlsanity.com/paper/2212.10888/) (2022)
- [A Unified Analysis of Mixed Sample Data Augmentation: A Loss Function Perspective](https://mlsanity.com/paper/2208.09913/) (2022)
- [Self-supervised Audio Teacher-Student Transformer for Both Clip-level and Frame-level Tasks](https://mlsanity.com/paper/2306.04186/) (2023)
- [VinTAGe: Joint Video and Text Conditioning for Holistic Audio Generation](https://mlsanity.com/paper/2412.10768/) (2024)
- [An overview of mixing augmentation methods and augmentation strategies](https://mlsanity.com/paper/2107.09887/) (2021)
- [animal2vec and MeerKAT: A self-supervised transformer for rare-event raw audio input and a large-scale reference dataset for bioacoustics](https://mlsanity.com/paper/2406.01253/) (2024)
- [ASiT: Local-Global Audio Spectrogram vIsion Transformer for Event Classification](https://mlsanity.com/paper/2211.13189/) (2022)
- [GenLabel: Mixup Relabeling using Generative Models](https://mlsanity.com/paper/2201.02354/) (2022)
- [Multi-view Audio and Music Classification](https://mlsanity.com/paper/2103.02420/) (2021)
- [Human-in-the-Loop Mixup](https://mlsanity.com/paper/2211.01202/) (2022)
- [Wider or Deeper Neural Network Architecture for Acoustic Scene Classification with Mismatched Recording Devices](https://mlsanity.com/paper/2203.12314/) (2022)
- [Environmental Sound Classification on the Edge: A Pipeline for Deep Acoustic Networks on Extremely Resource-Constrained Devices](https://mlsanity.com/paper/2103.03483/) (2021)
- [An Audio-Visual Dataset and Deep Learning Frameworks for Crowded Scene Classification](https://mlsanity.com/paper/2112.09172/) (2021)
- [A dataset for Audio-Visual Sound Event Detection in Movies](https://mlsanity.com/paper/2302.07315/) (2023)
- [SoundCLR: Contrastive Learning of Representations For Improved Environmental Sound Classification](https://mlsanity.com/paper/2103.01929/) (2021)
- [Label-Occurrence-Balanced Mixup for Long-tailed Recognition](https://mlsanity.com/paper/2110.04964/) (2021)
- [FlexiAST: Flexibility is What AST Needs](https://mlsanity.com/paper/2307.09286/) (2023)
- [Deep Epidemiological Modeling by Black-box Knowledge Distillation: An Accurate Deep Learning Model for COVID-19](https://mlsanity.com/paper/2101.10280/) (2021)
- [Midpoint Regularization: from High Uncertainty Training to Conservative Classification](https://mlsanity.com/paper/2106.13913/) (2021)
- [Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-devices](https://mlsanity.com/paper/2108.06128/) (2021)

## Abstract (arXiv)

Deep learning methods have achieved high performance in sound recognition tasks. Deciding how to feed the training data is important for further performance improvement. We propose a novel learning method for deep sound recognition: Between-Class learning (BC learning). Our strategy is to learn a discriminative feature space by recognizing the between-class sounds as between-class sounds. We generate between-class sounds by mixing two sounds belonging to different classes with a random ratio. We then input the mixed sound to the model and train the model to output the mixing ratio. The advantages of BC learning are not limited only to the increase in variation of the training data; BC learning leads to an enlargement of Fisher's criterion in the feature space and a regularization of the positional relationship among the feature distributions of the classes. The experimental results show that BC learning improves the performance on various sound recognition networks, datasets, and data augmentation schemes, in which BC learning proves to be always beneficial. Furthermore, we construct a new deep sound recognition network (EnvNet-v2) and train it with BC learning. As a result, we achieved a performance surpasses the human level.