# Llama: A Heterogeneous & Serverless Framework for Auto-Tuning Video Analytics Pipelines

arXiv: https://arxiv.org/abs/2102.01887 · HTML: https://mlsanity.com/paper/2102.01887/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2021.

## Built on by (most cited first, 16 of 16)

- [Teola: Towards End-to-End Optimization of LLM-based Applications](https://mlsanity.com/paper/2407.00326/) (2024)
- [Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platforms](https://mlsanity.com/paper/2508.18298/) (2025)
- [Serving and Optimizing Machine Learning Workflows on Heterogeneous Infrastructures](https://mlsanity.com/paper/2205.04713/) (2022)
- [Edge Video Analytics: A Survey on Applications, Systems and Enabling Techniques](https://mlsanity.com/paper/2211.15751/) (2022)
- [Extract-Transform-Load for Video Streams](https://mlsanity.com/paper/2310.04830/) (2023)
- [KAIROS: Building Cost-Efficient Machine Learning Inference Systems with Heterogeneous Cloud Resources](https://mlsanity.com/paper/2210.05889/) (2022)
- [Symphony: Optimized DNN Model Serving using Deferred Batch Scheduling](https://mlsanity.com/paper/2308.07470/) (2023)
- [IPA: Inference Pipeline Adaptation to Achieve High Accuracy and Cost-Efficiency](https://mlsanity.com/paper/2308.12871/) (2023)
- [A Survey on Video Analytics in Cloud-Edge-Terminal Collaborative Systems](https://mlsanity.com/paper/2502.06581/) (2025)
- [RLHFless: Serverless Computing for Efficient RLHF](https://mlsanity.com/paper/2602.22718/) (2026)
- [VQPy: An Object-Oriented Approach to Modern Video Analytics](https://mlsanity.com/paper/2311.01623/) (2023)
- [Rethinking Inference Placement for Deep Learning across Edge and Cloud Platforms: A Multi-Objective Optimization Perspective and Future Directions](https://mlsanity.com/paper/2510.22909/) (2025)
- [Optimizing video analytics inference pipelines: a case study](https://mlsanity.com/paper/2512.07009/) (2025)
- [StreamWise: Serving Multi-Modal Generation in Real-Time at Scale](https://mlsanity.com/paper/2603.05800/) (2026)
- [VineLM: Trie-Based Fine-Grained Control for Agentic Workflows](https://mlsanity.com/paper/2605.23914/) (2026)
- [Atlas: Optimizing Deployment of Compound AI Workflows on Heterogeneous Clusters](https://mlsanity.com/paper/2609.04513/) (2026)

## Abstract (arXiv)

The proliferation of camera-enabled devices and large video repositories has led to a diverse set of video analytics applications. These applications rely on video pipelines, represented as DAGs of operations, to transform videos, process extracted metadata, and answer questions like, "Is this intersection congested?" The latency and resource efficiency of pipelines can be optimized using configurable knobs for each operation (e.g., sampling rate, batch size, or type of hardware used). However, determining efficient configurations is challenging because (a) the configuration search space is exponentially large, and (b) the optimal configuration depends on users' desired latency and cost targets, (c) input video contents may exercise different paths in the DAG and produce a variable amount intermediate results. Existing video analytics and processing systems leave it to the users to manually configure operations and select hardware resources. We present Llama: a heterogeneous and serverless framework for auto-tuning video pipelines. Given an end-to-end latency target, Llama optimizes for cost efficiency by (a) calculating a latency target for each operation invocation, and (b) dynamically running a cost-based optimizer to assign configurations across heterogeneous hardware that best meet the calculated per-invocation latency target. This makes the problem of auto-tuning large video pipelines tractable and allows us to handle input-dependent behavior, conditional branches in the DAG, and execution variability. We describe the algorithms in Llama and evaluate it on a cloud platform using serverless CPU and GPU resources. We show that compared to state-of-the-art cluster and serverless video analytics and processing systems, Llama achieves 7.8x lower latency and 16x cost reduction on average.