# The streaming rollout of deep networks - towards fully model-parallel execution

arXiv: https://arxiv.org/abs/1806.04965 · HTML: https://mlsanity.com/paper/1806.04965/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2018.

## Built on by (most cited first, 5 of 5)

- [Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision](https://mlsanity.com/paper/2112.03423/) (2021)
- [Handling Delay in Real-Time Reinforcement Learning](https://mlsanity.com/paper/2503.23478/) (2025)
- [Improving Anytime Prediction with Parallel Cascaded Networks and a Temporal-Difference Loss](https://mlsanity.com/paper/2102.09808/) (2021)
- [Gradient Forward-Propagation for Large-Scale Temporal Video Modelling](https://mlsanity.com/paper/2106.08318/) (2021)
- [ItNet: iterative neural networks with small graphs for accurate, efficient and anytime semantic segmentation](https://mlsanity.com/paper/2101.08685/) (2021)

## Abstract (arXiv)

Deep neural networks, and in particular recurrent networks, are promising candidates to control autonomous agents that interact in real-time with the physical world. However, this requires a seamless integration of temporal features into the network's architecture. For the training of and inference with recurrent neural networks, they are usually rolled out over time, and different rollouts exist. Conventionally during inference, the layers of a network are computed in a sequential manner resulting in sparse temporal integration of information and long response times. In this study, we present a theoretical framework to describe rollouts, the level of model-parallelization they induce, and demonstrate differences in solving specific tasks. We prove that certain rollouts, also for networks with only skip and no recurrent connections, enable earlier and more frequent responses, and show empirically that these early responses have better performance. The streaming rollout maximizes these properties and enables a fully parallel execution of the network reducing runtime on massively parallel devices. Finally, we provide an open-source toolbox to design, train, evaluate, and interact with streaming rollouts.