# Sonic: A Sampling-based Online Controller for Streaming Applications

arXiv: https://arxiv.org/abs/2108.10701 · HTML: https://mlsanity.com/paper/2108.10701/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 62 references that are in the map.
- First posted on arXiv 15 Aug 2021.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Scalable Bayesian Optimization Using Deep Neural Networks](https://mlsanity.com/paper/1502.05700/) (2015)
- [Neural Architecture Search with Bayesian Optimisation and Optimal Transport](https://mlsanity.com/paper/1802.07191/) (2018)
- [Bayesian Optimization with Unknown Constraints](https://mlsanity.com/paper/1403.5607/) (2014)
- [NeuroVectorizer: End-to-End Vectorization with Deep Reinforcement Learning](https://mlsanity.com/paper/1909.13639/) (2019)
- [Safe Contextual Bayesian Optimization for Sustainable Room Temperature PID Control Tuning](https://mlsanity.com/paper/1906.12086/) (2019)
- [Constrained Bayesian Optimization for Automatic Chemical Design](https://mlsanity.com/paper/1709.05501/) (2017)
- [Data-efficient Auto-tuning with Bayesian Optimization: An Industrial Control Study](https://mlsanity.com/paper/1812.06325/) (2018)
- [Deep Kernels for Optimizing Locomotion Controllers](https://mlsanity.com/paper/1707.09062/) (2017)
- [Rapid Bayesian optimisation for synthesis of short polymer fiber materials](https://mlsanity.com/paper/1802.05841/) (2018)
- [SLAMBooster: An Application-aware Controller for Approximation in SLAM](https://mlsanity.com/paper/1811.01516/) (2018)
- [Capri: A Control System for Approximate Programs](https://mlsanity.com/paper/1706.00767/) (2017)
- [Compiler-Guided Throughput Scheduling for Many-core Machines](https://mlsanity.com/paper/2103.06647/) (2021)

## Abstract (arXiv)

Many applications in important problem domains such as machine learning and computer vision are streaming applications that take a sequence of inputs over time. It is challenging to find knob settings that optimize the run-time performance of such applications because the optimal knob settings are usually functions of inputs, computing platforms, time as well as user's requirements, which can be very diverse. Most prior works address this problem by offline profiling followed by training models for control. However, profiling-based approaches incur large overhead before execution; it is also difficult to redeploy them in other run-time configurations. In this paper, we propose Sonic, a sampling-based online controller for long-running streaming applications that does not require profiling ahead of time. Within each phase of a streaming application's execution, Sonic utilizes the beginning portion to sample the knob space strategically and aims to pick the optimal knob setting for the rest of the phase, given a user-specified constrained optimization problem. A hybrid approach of machine learning regressions and Bayesian optimization are used for better overall sampling choices. Sonic is implemented independent of application, device, input, performance objective and constraints. We evaluate Sonic on traditional parallel benchmarks as well as on deep learning inference benchmarks across multiple platforms. Our experiments show that when using Sonic to control knob settings, application run-time performance is only 5.3% less than if optimal knob settings were used, demonstrating that Sonic is able to find near-optimal knob settings under diverse run-time configurations without prior knowledge quickly.