# AutoTransfer: Subject Transfer Learning with Censored Representations on Biosignals Data

arXiv: https://arxiv.org/abs/2112.09796 · HTML: https://mlsanity.com/paper/2112.09796/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 17 Dec 2021.

## Builds on (most cited first, 24 of 24)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)
- [Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes](https://mlsanity.com/paper/1906.07697/) (2019)
- [BEGAN: Boundary Equilibrium Generative Adversarial Networks](https://mlsanity.com/paper/1703.10717/) (2017)
- [Learning feed-forward one-shot learners](https://mlsanity.com/paper/1606.05233/) (2016)
- [Gradient Estimators for Implicit Models](https://mlsanity.com/paper/1705.07107/) (2017)
- [A Spectral Approach to Gradient Estimation for Implicit Distributions](https://mlsanity.com/paper/1806.02925/) (2018)
- [Nonparametric Score Estimators](https://mlsanity.com/paper/2005.10099/) (2020)
- [Mutual Information Gradient Estimation for Representation Learning](https://mlsanity.com/paper/2005.01123/) (2020)
- [Transfer Learning in Brain-Computer Interfaces with Adversarial Variational Autoencoders](https://mlsanity.com/paper/1812.06857/) (2018)
- [Disentangled Adversarial Autoencoder for Subject-Invariant Physiological Feature Extraction](https://mlsanity.com/paper/2008.11426/) (2020)
- [Adversarial Deep Learning in EEG Biometrics](https://mlsanity.com/paper/1903.11673/) (2019)
- [Universal Physiological Representation Learning with Soft-Disentangled Rateless Autoencoders](https://mlsanity.com/paper/2009.13453/) (2020)
- [Invariant Representations from Adversarially Censored Autoencoders](https://mlsanity.com/paper/1805.08097/) (2018)
- [Disentangled Adversarial Transfer Learning for Physiological Biosignals](https://mlsanity.com/paper/2004.08289/) (2020)

## Built on by (most cited first, 3 of 3)

- [quEEGNet: Quantum AI for Biosignal Processing](https://mlsanity.com/paper/2210.00864/) (2022)
- [Stabilizing Subject Transfer in EEG Classification with Divergence Estimation](https://mlsanity.com/paper/2310.08762/) (2023)
- [Fast and Expressive Gesture Recognition using a Combination-Homomorphic Electromyogram Encoder](https://mlsanity.com/paper/2311.14675/) (2023)

## Abstract (arXiv)

We provide a regularization framework for subject transfer learning in which we seek to train an encoder and classifier to minimize classification loss, subject to a penalty measuring independence between the latent representation and the subject label. We introduce three notions of independence and corresponding penalty terms using mutual information or divergence as a proxy for independence. For each penalty term, we provide several concrete estimation algorithms, using analytic methods as well as neural critic functions. We provide a hands-off strategy for applying this diverse family of regularization algorithms to a new dataset, which we call "AutoTransfer". We evaluate the performance of these individual regularization strategies and our AutoTransfer method on EEG, EMG, and ECoG datasets, showing that these approaches can improve subject transfer learning for challenging real-world datasets.