# Transtreaming: Adaptive Delay-aware Transformer for Real-time Streaming Perception

arXiv: https://arxiv.org/abs/2409.06584 · HTML: https://mlsanity.com/paper/2409.06584/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 23 references that are in the map.
- First posted on arXiv 10 Sep 2024.

## Builds on (most cited first, 18 of 18)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [Deep Feature Flow for Video Recognition](https://mlsanity.com/paper/1611.07715/) (2016)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [Detect to Track and Track to Detect](https://mlsanity.com/paper/1710.03958/) (2017)
- [Sequence Level Semantics Aggregation for Video Object Detection](https://mlsanity.com/paper/1907.06390/) (2019)
- [Relation Distillation Networks for Video Object Detection](https://mlsanity.com/paper/1908.09511/) (2019)
- [Towards High Performance Video Object Detection](https://mlsanity.com/paper/1711.11577/) (2017)
- [Video Object Detection with an Aligned Spatial-Temporal Memory](https://mlsanity.com/paper/1712.06317/) (2017)
- [Mobile Video Object Detection with Temporally-Aware Feature Maps](https://mlsanity.com/paper/1711.06368/) (2017)
- [Real-time Object Detection for Streaming Perception](https://mlsanity.com/paper/2203.12338/) (2022)
- [DAMO-StreamNet: Optimizing Streaming Perception in Autonomous Driving](https://mlsanity.com/paper/2303.17144/) (2023)

## Built on by (most cited first, 3 of 3)

- [4DSegStreamer: Streaming 4D Panoptic Segmentation via Dual Threads](https://mlsanity.com/paper/2510.17664/) (2025)
- [Bend the Clock: Predicting Ahead to Beat Latency in Event-Based Object Detection](https://mlsanity.com/paper/2609.26919/) (2026)
- [Rethinking Streaming-Perception Evaluation on Heterogeneous Edge Platforms](https://mlsanity.com/paper/2610.05578/) (2026)

## Abstract (arXiv)

Real-time object detection is critical for the decision-making process for many real-world applications, such as collision avoidance and path planning in autonomous driving. This work presents an innovative real-time streaming perception method, Transtreaming, which addresses the challenge of real-time object detection with dynamic computational delay. The core innovation of Transtreaming lies in its adaptive delay-aware transformer, which can concurrently predict multiple future frames and select the output that best matches the real-world present time, compensating for any system-induced computation delays. The proposed model outperforms the existing state-of-the-art methods, even in single-frame detection scenarios, by leveraging a transformer-based methodology. It demonstrates robust performance across a range of devices, from powerful V100 to modest 2080Ti, achieving the highest level of perceptual accuracy on all platforms. Unlike most state-of-the-art methods that struggle to complete computation within a single frame on less powerful devices, Transtreaming meets the stringent real-time processing requirements on all kinds of devices. The experimental results emphasize the system's adaptability and its potential to significantly improve the safety and reliability for many real-world systems, such as autonomous driving.