# Real-Time Machine Learning: The Missing Pieces

arXiv: https://arxiv.org/abs/1703.03924 · HTML: https://mlsanity.com/paper/1703.03924/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Mar 2017.

## Built on by (most cited first, 7 of 7)

- [A Multi-Stage Automated Online Network Data Stream Analytics Framework for IIoT Systems](https://mlsanity.com/paper/2210.01985/) (2022)
- [Time-sensitive Learning for Heterogeneous Federated Edge Intelligence](https://mlsanity.com/paper/2301.10977/) (2023)
- [AutoML in Heavily Constrained Applications](https://mlsanity.com/paper/2306.16913/) (2023)
- [A Trainable Approach to Zero-delay Smoothing Spline Interpolation](https://mlsanity.com/paper/2203.03776/) (2022)
- [CapillaryX: A Software Design Pattern for Analyzing Medical Images in Real-time using Deep Learning](https://mlsanity.com/paper/2204.08462/) (2022)
- [Machines & Influence: An Information Systems Lens](https://mlsanity.com/paper/2111.13365/) (2021)
- [Keep Your Friends Close: Leveraging Affinity Groups to Accelerate AI Inference Workflows](https://mlsanity.com/paper/2312.11488/) (2023)

## Abstract (arXiv)

Machine learning applications are increasingly deployed not only to serve predictions using static models, but also as tightly-integrated components of feedback loops involving dynamic, real-time decision making. These applications pose a new set of requirements, none of which are difficult to achieve in isolation, but the combination of which creates a challenge for existing distributed execution frameworks: computation with millisecond latency at high throughput, adaptive construction of arbitrary task graphs, and execution of heterogeneous kernels over diverse sets of resources. We assert that a new distributed execution framework is needed for such ML applications and propose a candidate approach with a proof-of-concept architecture that achieves a 63x performance improvement over a state-of-the-art execution framework for a representative application.