# Aux-Drop: Handling Haphazard Inputs in Online Learning Using Auxiliary Dropouts

arXiv: https://arxiv.org/abs/2303.05155 · HTML: https://mlsanity.com/paper/2303.05155/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 65 references that are in the map.
- First posted on arXiv 9 Mar 2023.

## Builds on (most cited first, 23 of 23)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [A Brief Review of Domain Adaptation](https://mlsanity.com/paper/2010.03978/) (2020)
- [Efficient Object Localization Using Convolutional Networks](https://mlsanity.com/paper/1411.4280/) (2014)
- [Online Learning: A Comprehensive Survey](https://mlsanity.com/paper/1802.02871/) (2018)
- [Stochastic Activation Pruning for Robust Adversarial Defense](https://mlsanity.com/paper/1803.01442/) (2018)
- [Online Deep Learning: Learning Deep Neural Networks on the Fly](https://mlsanity.com/paper/1711.03705/) (2017)
- [Learning with a Strong Adversary](https://mlsanity.com/paper/1511.03034/) (2015)
- [Open-environment Machine Learning](https://mlsanity.com/paper/2206.00423/) (2022)
- [Open-world Machine Learning: Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2105.13448/) (2021)
- [Learning with Feature Evolvable Streams](https://mlsanity.com/paper/1706.05259/) (2017)
- [Autonomous Deep Learning: Continual Learning Approach for Dynamic Environments](https://mlsanity.com/paper/1810.07348/) (2018)
- [Guided Dropout](https://mlsanity.com/paper/1812.03965/) (2018)
- [One-Pass Learning with Incremental and Decremental Features](https://mlsanity.com/paper/1605.09082/) (2016)
- [Nonstationary data stream classification with online active learning and siamese neural networks](https://mlsanity.com/paper/2210.01090/) (2022)
- [Prediction with Unpredictable Feature Evolution](https://mlsanity.com/paper/1904.12171/) (2019)
- [Auxiliary Network: Scalable and agile online learning for dynamic system with inconsistently available inputs](https://mlsanity.com/paper/2008.11828/) (2020)
- [Evolving Metric Learning for Incremental and Decremental Features](https://mlsanity.com/paper/2006.15334/) (2020)
- [How Does GAN-based Semi-supervised Learning Work?](https://mlsanity.com/paper/2007.05692/) (2020)

## Built on by (most cited first, 5 of 5)

- [Online Learning under Haphazard Input Conditions: A Comprehensive Review and Analysis](https://mlsanity.com/paper/2404.04903/) (2024)
- [MODL: Multilearner Online Deep Learning](https://mlsanity.com/paper/2405.18281/) (2024)
- [Hedging Is Not All You Need: A Simple Baseline for Online Learning Under Haphazard Inputs](https://mlsanity.com/paper/2409.10242/) (2024)
- [packetLSTM: Dynamic LSTM Framework for Streaming Data with Varying Feature Space](https://mlsanity.com/paper/2410.17394/) (2024)
- [Haphazard Inputs as Images in Online Learning](https://mlsanity.com/paper/2504.02912/) (2025)

## Abstract (arXiv)

Many real-world applications based on online learning produce streaming data that is haphazard in nature, i.e., contains missing features, features becoming obsolete in time, the appearance of new features at later points in time and a lack of clarity on the total number of input features. These challenges make it hard to build a learnable system for such applications, and almost no work exists in deep learning that addresses this issue. In this paper, we present Aux-Drop, an auxiliary dropout regularization strategy for online learning that handles the haphazard input features in an effective manner. Aux-Drop adapts the conventional dropout regularization scheme for the haphazard input feature space ensuring that the final output is minimally impacted by the chaotic appearance of such features. It helps to prevent the co-adaptation of especially the auxiliary and base features, as well as reduces the strong dependence of the output on any of the auxiliary inputs of the model. This helps in better learning for scenarios where certain features disappear in time or when new features are to be modelled. The efficacy of Aux-Drop has been demonstrated through extensive numerical experiments on SOTA benchmarking datasets that include Italy Power Demand, HIGGS, SUSY and multiple UCI datasets. The code is available at https://github.com/Rohit102497/Aux-Drop.