# SubSpectral Normalization for Neural Audio Data Processing

arXiv: https://arxiv.org/abs/2103.13620 · HTML: https://mlsanity.com/paper/2103.13620/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 21 references that are in the map.
- First posted on arXiv 25 Mar 2021.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification](https://mlsanity.com/paper/1608.04363/) (2016)
- [A multi-device dataset for urban acoustic scene classification](https://mlsanity.com/paper/1807.09840/) (2018)
- [Deep Residual Learning for Small-Footprint Keyword Spotting](https://mlsanity.com/paper/1710.10361/) (2017)
- [Temporal Convolution for Real-time Keyword Spotting on Mobile Devices](https://mlsanity.com/paper/1904.03814/) (2019)
- [Advances in Joint CTC-Attention based End-to-End Speech Recognition with a Deep CNN Encoder and RNN-LM](https://mlsanity.com/paper/1706.02737/) (2017)
- [The Receptive Field as a Regularizer in Deep Convolutional Neural Networks for Acoustic Scene Classification](https://mlsanity.com/paper/1907.01803/) (2019)
- [SubSpectralNet - Using Sub-Spectrogram based Convolutional Neural Networks for Acoustic Scene Classification](https://mlsanity.com/paper/1810.12642/) (2018)
- [EdgeSpeechNets: Highly Efficient Deep Neural Networks for Speech Recognition on the Edge](https://mlsanity.com/paper/1810.08559/) (2018)
- [Sub-band Convolutional Neural Networks for Small-footprint Spoken Term Classification](https://mlsanity.com/paper/1907.01448/) (2019)

## Built on by (most cited first, 11 of 11)

- [Broadcasted Residual Learning for Efficient Keyword Spotting](https://mlsanity.com/paper/2106.04140/) (2021)
- [QTI Submission to DCASE 2021: residual normalization for device-imbalanced acoustic scene classification with efficient design](https://mlsanity.com/paper/2206.13909/) (2022)
- [Domain Generalization on Efficient Acoustic Scene Classification using Residual Normalization](https://mlsanity.com/paper/2111.06531/) (2021)
- [Phase-driven Domain Generalizable Learning for Nonstationary Time Series](https://mlsanity.com/paper/2402.05960/) (2024)
- [Avoid Overfitting User Specific Information in Federated Keyword Spotting](https://mlsanity.com/paper/2206.08864/) (2022)
- [Real-Time Streamable Generative Speech Restoration with Flow Matching](https://mlsanity.com/paper/2512.19442/) (2025)
- [Real-Time Streaming Mel Vocoding with Generative Flow Matching](https://mlsanity.com/paper/2509.15085/) (2025)
- [A Temporal-oriented Broadcast ResNet for COVID-19 Detection](https://mlsanity.com/paper/2203.17012/) (2022)
- [Advancing Airport Tower Command Recognition: Integrating Squeeze-and-Excitation and Broadcasted Residual Learning](https://mlsanity.com/paper/2406.18313/) (2024)
- [Offload Rethinking by Cloud Assistance for Efficient Environmental Sound Recognition on LPWANs](https://mlsanity.com/paper/2502.15285/) (2025)
- [Efficient Neural and Numerical Methods for High-Quality Online Speech Spectrogram Inversion via Gradient Theorem](https://mlsanity.com/paper/2505.24498/) (2025)

## Abstract (arXiv)

Convolutional Neural Networks are widely used in various machine learning domains. In image processing, the features can be obtained by applying 2D convolution to all spatial dimensions of the input. However, in the audio case, frequency domain input like Mel-Spectrogram has different and unique characteristics in the frequency dimension. Thus, there is a need for a method that allows the 2D convolution layer to handle the frequency dimension differently. In this work, we introduce SubSpectral Normalization (SSN), which splits the input frequency dimension into several groups (sub-bands) and performs a different normalization for each group. SSN also includes an affine transformation that can be applied to each group. Our method removes the inter-frequency deflection while the network learns a frequency-aware characteristic. In the experiments with audio data, we observed that SSN can efficiently improve the network's performance.