# Hedging Is Not All You Need: A Simple Baseline for Online Learning Under Haphazard Inputs

arXiv: https://arxiv.org/abs/2409.10242 · HTML: https://mlsanity.com/paper/2409.10242/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 16 references that are in the map.
- First posted on arXiv 16 Sep 2024.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [A Generalist Agent](https://mlsanity.com/paper/2205.06175/) (2022)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [Online Deep Learning: Learning Deep Neural Networks on the Fly](https://mlsanity.com/paper/1711.03705/) (2017)
- [Regression Transformer: Concurrent sequence regression and generation for molecular language modeling](https://mlsanity.com/paper/2202.01338/) (2022)
- [Auxiliary Network: Scalable and agile online learning for dynamic system with inconsistently available inputs](https://mlsanity.com/paper/2008.11828/) (2020)
- [Aux-Drop: Handling Haphazard Inputs in Online Learning Using Auxiliary Dropouts](https://mlsanity.com/paper/2303.05155/) (2023)

## Built on by (most cited first, 1 of 1)

- [Dynamic Hedging Strategies in Derivatives Markets with LLM-Driven Sentiment and News Analytics](https://mlsanity.com/paper/2504.04295/) (2025)

## Abstract (arXiv)

Handling haphazard streaming data, such as data from edge devices, presents a challenging problem. Over time, the incoming data becomes inconsistent, with missing, faulty, or new inputs reappearing. Therefore, it requires models that are reliable. Recent methods to solve this problem depend on a hedging-based solution and require specialized elements like auxiliary dropouts, forked architectures, and intricate network design. We observed that hedging can be reduced to a special case of weighted residual connection; this motivated us to approximate it with plain self-attention. In this work, we propose HapNet, a simple baseline that is scalable, does not require online backpropagation, and is adaptable to varying input types. All present methods are restricted to scaling with a fixed window; however, we introduce a more complex problem of scaling with a variable window where the data becomes positionally uncorrelated, and cannot be addressed by present methods. We demonstrate that a variant of the proposed approach can work even for this complex scenario. We extensively evaluated the proposed approach on five benchmarks and found competitive performance.