# A Modulation Layer to Increase Neural Network Robustness Against Data Quality Issues

arXiv: https://arxiv.org/abs/2107.08574 · HTML: https://mlsanity.com/paper/2107.08574/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 66 references that are in the map.
- First posted on arXiv 19 Jul 2021.

## Builds on (most cited first, 17 of 17)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [GAIN: Missing Data Imputation using Generative Adversarial Nets](https://mlsanity.com/paper/1806.02920/) (2018)
- [Handling Missing Data with Graph Representation Learning](https://mlsanity.com/paper/2010.16418/) (2020)
- [Ethical Machine Learning in Health Care](https://mlsanity.com/paper/2009.10576/) (2020)
- [The Case for Bayesian Deep Learning](https://mlsanity.com/paper/2001.10995/) (2020)
- [Variational Autoencoder with Arbitrary Conditioning](https://mlsanity.com/paper/1806.02382/) (2018)
- [Towards Bayesian Deep Learning: A Framework and Some Existing Methods](https://mlsanity.com/paper/1608.06884/) (2016)
- [Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks](https://mlsanity.com/paper/1707.02476/) (2017)
- [Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity](https://mlsanity.com/paper/2002.10585/) (2020)
- [Feature-Wise Bias Amplification](https://mlsanity.com/paper/1812.08999/) (2018)
- [Robust learning from corrupted EEG with dynamic spatial filtering](https://mlsanity.com/paper/2105.12916/) (2021)
- [Multiple Imputation with Denoising Autoencoder using Metamorphic Truth and Imputation Feedback](https://mlsanity.com/paper/2002.08338/) (2020)
- [A Factored Generalized Additive Model for Clinical Decision Support in the Operating Room](https://mlsanity.com/paper/1907.12596/) (2019)

## Built on by (most cited first, 1 of 1)

- [A Patient-Doctor-NLP-System to contest inequality for less privileged](https://mlsanity.com/paper/2512.06734/) (2025)

## Abstract (arXiv)

Data missingness and quality are common problems in machine learning, especially for high-stakes applications such as healthcare. Developers often train machine learning models on carefully curated datasets using only high quality data; however, this reduces the utility of such models in production environments. We propose a novel neural network modification to mitigate the impacts of low quality and missing data which involves replacing the fixed weights of a fully-connected layer with a function of an additional input. This is inspired from neuromodulation in biological neural networks where the cortex can up- and down-regulate inputs based on their reliability and the presence of other data. In testing, with reliability scores as a modulating signal, models with modulating layers were found to be more robust against degradation of data quality, including additional missingness. These models are superior to imputation as they save on training time by completely skipping the imputation process and further allow the introduction of other data quality measures that imputation cannot handle. Our results suggest that explicitly accounting for reduced information quality with a modulating fully connected layer can enable the deployment of artificial intelligence systems in real-time applications.