# Continual Inference: A Library for Efficient Online Inference with Deep Neural Networks in PyTorch

arXiv: https://arxiv.org/abs/2204.03418 · HTML: https://mlsanity.com/paper/2204.03418/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 7 Apr 2022.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [X3D: Expanding Architectures for Efficient Video Recognition](https://mlsanity.com/paper/2004.04730/) (2020)
- [Skeleton-based Action Recognition via Spatial and Temporal Transformer Networks](https://mlsanity.com/paper/2008.07404/) (2020)
- [Beyond Temporal Pooling: Recurrence and Temporal Convolutions for Gesture Recognition in Video](https://mlsanity.com/paper/1506.01911/) (2015)
- [Continual Transformers: Redundancy-Free Attention for Online Inference](https://mlsanity.com/paper/2201.06268/) (2022)
- [Continual 3D Convolutional Neural Networks for Real-time Processing of Videos](https://mlsanity.com/paper/2106.00050/) (2021)
- [Dissected 3D CNNs: Temporal Skip Connections for Efficient Online Video Processing](https://mlsanity.com/paper/2009.14639/) (2020)

## Built on by (most cited first, 8 of 8)

- [Efficient High-Resolution Deep Learning: A Survey](https://mlsanity.com/paper/2207.13050/) (2022)
- [Real-Time Streamable Generative Speech Restoration with Flow Matching](https://mlsanity.com/paper/2512.19442/) (2025)
- [Continual Low-Rank Scaled Dot-product Attention](https://mlsanity.com/paper/2412.03214/) (2024)
- [LOBERT: Generative AI Foundation Model for Limit Order Book Messages](https://mlsanity.com/paper/2511.12563/) (2025)
- [Online hand gesture recognition using Continual Graph Transformers](https://mlsanity.com/paper/2502.14939/) (2025)
- [Real-Time Streaming Mel Vocoding with Generative Flow Matching](https://mlsanity.com/paper/2509.15085/) (2025)
- [Efficient Online Processing with Deep Neural Networks](https://mlsanity.com/paper/2306.13474/) (2023)
- [DeepCoT: Deep Continual Transformers for Real-Time Inference on Data Streams](https://mlsanity.com/paper/2511.17693/) (2025)

## Abstract (arXiv)

We present Continual Inference, a Python library for implementing Continual Inference Networks (CINs) in PyTorch, a class of Neural Networks designed specifically for efficient inference in both online and batch processing scenarios. We offer a comprehensive introduction and guide to CINs and their implementation in practice, and provide best-practices and code examples for composing complex modules for modern Deep Learning. Continual Inference is readily downloadable via the Python Package Index and at urlwww.github.com/lukashedegaard/continual-inference.