# SOI: Scaling Down Computational Complexity by Estimating Partial States of the Model

arXiv: https://arxiv.org/abs/2410.03813 · HTML: https://mlsanity.com/paper/2410.03813/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 35 references that are in the map.
- First posted on arXiv 4 Oct 2024.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [GhostNet: More Features from Cheap Operations](https://mlsanity.com/paper/1911.11907/) (2019)
- [SkipNet: Learning Dynamic Routing in Convolutional Networks](https://mlsanity.com/paper/1711.09485/) (2017)
- [Adaptive Neural Networks for Efficient Inference](https://mlsanity.com/paper/1702.07811/) (2017)
- [Convolutional Networks with Adaptive Inference Graphs](https://mlsanity.com/paper/1711.11503/) (2017)
- [The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results](https://mlsanity.com/paper/2005.13981/) (2020)
- [Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks](https://mlsanity.com/paper/1708.06834/) (2017)
- [Super-Human Performance in Online Low-latency Recognition of Conversational Speech](https://mlsanity.com/paper/2010.03449/) (2020)
- [Learning to Skim Text](https://mlsanity.com/paper/1704.06877/) (2017)
- [Neural Speed Reading with Structural-Jump-LSTM](https://mlsanity.com/paper/1904.00761/) (2019)
- [Neural Speed Reading via Skim-RNN](https://mlsanity.com/paper/1711.02085/) (2017)
- [Variable Computation in Recurrent Neural Networks](https://mlsanity.com/paper/1611.06188/) (2016)
- [Short-Term Memory Convolutions](https://mlsanity.com/paper/2302.04331/) (2023)
- [Efficient Low-Latency Speech Enhancement with Mobile Audio Streaming Networks](https://mlsanity.com/paper/2008.07244/) (2020)

## Abstract (arXiv)

Consumer electronics used to follow the miniaturization trend described by Moore's Law. Despite increased processing power in Microcontroller Units (MCUs), MCUs used in the smallest appliances are still not capable of running even moderately big, state-of-the-art artificial neural networks (ANNs) especially in time-sensitive scenarios. In this work, we present a novel method called Scattered Online Inference (SOI) that aims to reduce the computational complexity of ANNs. SOI leverages the continuity and seasonality of time-series data and model predictions, enabling extrapolation for processing speed improvements, particularly in deeper layers. By applying compression, SOI generates more general inner partial states of ANN, allowing skipping full model recalculation at each inference.