# Cascade: A Platform for Delay-Sensitive Edge Intelligence

arXiv: https://arxiv.org/abs/2311.17329 · HTML: https://mlsanity.com/paper/2311.17329/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 63 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Builds on (most cited first, 5 of 5)

- [StrongSORT: Make DeepSORT Great Again](https://mlsanity.com/paper/2202.13514/) (2022)
- [From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting](https://mlsanity.com/paper/2012.01526/) (2020)
- [Learning Generalisable Omni-Scale Representations for Person Re-Identification](https://mlsanity.com/paper/1910.06827/) (2019)
- [Real-time Anomaly Detection and Classification in Streaming PMU Data](https://mlsanity.com/paper/1911.06316/) (2019)
- [Spindle: Techniques for Optimizing Atomic Multicast on RDMA](https://mlsanity.com/paper/2110.00886/) (2021)

## Built on by (most cited first, 1 of 1)

- [Keep Your Friends Close: Leveraging Affinity Groups to Accelerate AI Inference Workflows](https://mlsanity.com/paper/2312.11488/) (2023)

## Abstract (arXiv)

Interactive intelligent computing applications are increasingly prevalent, creating a need for AI/ML platforms optimized to reduce per-event latency while maintaining high throughput and efficient resource management. Yet many intelligent applications run on AI/ML platforms that optimize for high throughput even at the cost of high tail-latency. Cascade is a new AI/ML hosting platform intended to untangle this puzzle. Innovations include a legacy-friendly storage layer that moves data with minimal copying and a "fast path" that collocates data and computation to maximize responsiveness. Our evaluation shows that Cascade reduces latency by orders of magnitude with no loss of throughput.