# CADDA: Class-wise Automatic Differentiable Data Augmentation for EEG Signals

arXiv: https://arxiv.org/abs/2106.13695 · HTML: https://mlsanity.com/paper/2106.13695/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 57 references that are in the map.
- First posted on arXiv 25 Jun 2021.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Fast AutoAugment](https://mlsanity.com/paper/1905.00397/) (2019)
- [Deep learning-based electroencephalography analysis: a systematic review](https://mlsanity.com/paper/1901.05498/) (2019)
- [Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules](https://mlsanity.com/paper/1905.05393/) (2019)
- [Gradient Estimation Using Stochastic Computation Graphs](https://mlsanity.com/paper/1506.05254/) (2015)
- [Adversarial AutoAugment](https://mlsanity.com/paper/1912.11188/) (2019)
- [Backpropagation through the Void: Optimizing control variates for black-box gradient estimation](https://mlsanity.com/paper/1711.00123/) (2017)
- [Faster AutoAugment: Learning Augmentation Strategies using Backpropagation](https://mlsanity.com/paper/1911.06987/) (2019)
- [U-Time: A Fully Convolutional Network for Time Series Segmentation Applied to Sleep Staging](https://mlsanity.com/paper/1910.11162/) (2019)
- [Subject-Aware Contrastive Learning for Biosignals](https://mlsanity.com/paper/2007.04871/) (2020)
- [A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series](https://mlsanity.com/paper/1707.03321/) (2017)
- [DADA: Differentiable Automatic Data Augmentation](https://mlsanity.com/paper/2003.03780/) (2020)
- [EEG-GAN: Generative adversarial networks for electroencephalograhic (EEG) brain signals](https://mlsanity.com/paper/1806.01875/) (2018)
- [Online Hyper-parameter Learning for Auto-Augmentation Strategy](https://mlsanity.com/paper/1905.07373/) (2019)
- [Machine-Learning-Based Diagnostics of EEG Pathology](https://mlsanity.com/paper/2002.05115/) (2020)
- [SalientSleepNet: Multimodal Salient Wave Detection Network for Sleep Staging](https://mlsanity.com/paper/2105.13864/) (2021)
- [Improving Auto-Augment via Augmentation-Wise Weight Sharing](https://mlsanity.com/paper/2009.14737/) (2020)
- [Dreaming More Data: Class-dependent Distributions over Diffeomorphisms for Learned Data Augmentation](https://mlsanity.com/paper/1510.02795/) (2015)
- [Learning from Heterogeneous EEG Signals with Differentiable Channel Reordering](https://mlsanity.com/paper/2010.13694/) (2020)
- [Addressing Class Imbalance in Classification Problems of Noisy Signals by using Fourier Transform Surrogates](https://mlsanity.com/paper/1806.08675/) (2018)
- [Adaptive Weighting Scheme for Automatic Time-Series Data Augmentation](https://mlsanity.com/paper/2102.08310/) (2021)
- [HAMLET: Interpretable Human And Machine co-LEarning Technique](https://mlsanity.com/paper/1803.09702/) (2018)

## Built on by (most cited first, 22 of 22)

- [A framework for bilevel optimization that enables stochastic and global variance reduction algorithms](https://mlsanity.com/paper/2201.13409/) (2022)
- [Data augmentation for learning predictive models on EEG: a systematic comparison](https://mlsanity.com/paper/2206.14483/) (2022)
- [Optimal Algorithms for Stochastic Bilevel Optimization under Relaxed Smoothness Conditions](https://mlsanity.com/paper/2306.12067/) (2023)
- [CUDA: Curriculum of Data Augmentation for Long-Tailed Recognition](https://mlsanity.com/paper/2302.05499/) (2023)
- [A General Theory of Correct, Incorrect, and Extrinsic Equivariance](https://mlsanity.com/paper/2303.04745/) (2023)
- [A Lower Bound and a Near-Optimal Algorithm for Bilevel Empirical Risk Minimization](https://mlsanity.com/paper/2302.08766/) (2023)
- [Parametric Augmentation for Time Series Contrastive Learning](https://mlsanity.com/paper/2402.10434/) (2024)
- [Deep invariant networks with differentiable augmentation layers](https://mlsanity.com/paper/2202.02142/) (2022)
- [Is Limited Participant Diversity Impeding EEG-based Machine Learning?](https://mlsanity.com/paper/2503.13497/) (2025)
- [AimTS: Augmented Series and Image Contrastive Learning for Time Series Classification](https://mlsanity.com/paper/2504.09993/) (2025)
- [Geometric Neural Network based on Phase Space for BCI-EEG decoding](https://mlsanity.com/paper/2403.05645/) (2024)
- [Impact of dataset size and long-term ECoG-based BCI usage on deep learning decoders performance](https://mlsanity.com/paper/2209.03789/) (2022)
- [Empirical Evaluation of Data Augmentations for Biobehavioral Time Series Data with Deep Learning](https://mlsanity.com/paper/2210.06701/) (2022)
- [Review of Deep Representation Learning Techniques for Brain-Computer Interfaces and Recommendations](https://mlsanity.com/paper/2405.19345/) (2024)
- [MVCNet: Multi-View Contrastive Network for Motor Imagery Classification](https://mlsanity.com/paper/2502.17482/) (2025)
- [CAAP: Class-Dependent Automatic Data Augmentation Based On Adaptive Policies For Time Series](https://mlsanity.com/paper/2404.00898/) (2024)
- [Exploring the Generalization Capabilities of AID-based Bi-level Optimization](https://mlsanity.com/paper/2411.16081/) (2024)
- [FEMBA on the Edge: Physiologically-Aware Pre-Training, Quantization, and Deployment of a Bidirectional Mamba EEG Foundation Model on an Ultra-low Power Microcontroller](https://mlsanity.com/paper/2603.26716/) (2026)
- [Distributionally Robust Cross Subject EEG Decoding](https://mlsanity.com/paper/2308.11651/) (2023)
- [Meta-Prior: Meta learning for Adaptive Inverse Problem Solvers](https://mlsanity.com/paper/2311.18710/) (2023)
- [Class-specific Data Augmentation for Plant Stress Classification](https://mlsanity.com/paper/2406.13081/) (2024)
- [A swap-adversarial framework for improving domain generalization in electrocorticography-based Parkinson's disease classification](https://mlsanity.com/paper/2602.10528/) (2026)

## Abstract (arXiv)

Data augmentation is a key element of deep learning pipelines, as it informs the network during training about transformations of the input data that keep the label unchanged. Manually finding adequate augmentation methods and parameters for a given pipeline is however rapidly cumbersome. In particular, while intuition can guide this decision for images, the design and choice of augmentation policies remains unclear for more complex types of data, such as neuroscience signals. Besides, class-dependent augmentation strategies have been surprisingly unexplored in the literature, although it is quite intuitive: changing the color of a car image does not change the object class to be predicted, but doing the same to the picture of an orange does. This paper investigates gradient-based automatic data augmentation algorithms amenable to class-wise policies with exponentially larger search spaces. Motivated by supervised learning applications using EEG signals for which good augmentation policies are mostly unknown, we propose a new differentiable relaxation of the problem. In the class-agnostic setting, results show that our new relaxation leads to optimal performance with faster training than competing gradient-based methods, while also outperforming gradient-free methods in the class-wise setting. This work proposes also novel differentiable augmentation operations relevant for sleep stage classification.