# Learning Complexity-Aware Cascades for Deep Pedestrian Detection

arXiv: https://arxiv.org/abs/1507.05348 · HTML: https://mlsanity.com/paper/1507.05348/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jul 2015.

## Built on by (most cited first, 18 of 18)

- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [A Declarative System for Optimizing AI Workloads](https://mlsanity.com/paper/2405.14696/) (2024)
- [GEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge](https://mlsanity.com/paper/2201.07705/) (2022)
- [Track without Appearance: Learn Box and Tracklet Embedding with Local and Global Motion Patterns for Vehicle Tracking](https://mlsanity.com/paper/2108.06029/) (2021)
- [Efficient Online ML API Selection for Multi-Label Classification Tasks](https://mlsanity.com/paper/2102.09127/) (2021)
- [Boggart: Towards General-Purpose Acceleration of Retrospective Video Analytics](https://mlsanity.com/paper/2106.15315/) (2021)
- [Multiple Object Tracking with Mixture Density Networks for Trajectory Estimation](https://mlsanity.com/paper/2106.10950/) (2021)
- [Pedestrian Detection: Domain Generalization, CNNs, Transformers and Beyond](https://mlsanity.com/paper/2201.03176/) (2022)
- [MadEye: Boosting Live Video Analytics Accuracy with Adaptive Camera Configurations](https://mlsanity.com/paper/2304.02101/) (2023)
- [T-RECX: Tiny-Resource Efficient Convolutional neural networks with early-eXit](https://mlsanity.com/paper/2207.06613/) (2022)
- [FrameHopper: Selective Processing of Video Frames in Detection-driven Real-Time Video Analytics](https://mlsanity.com/paper/2203.11493/) (2022)
- [Transformer-based assignment decision network for multiple object tracking](https://mlsanity.com/paper/2208.03571/) (2022)
- [UnfoldML: Cost-Aware and Uncertainty-Based Dynamic 2D Prediction for Multi-Stage Classification](https://mlsanity.com/paper/2210.15056/) (2022)
- [Which to Match? Selecting Consistent GT-Proposal Assignment for Pedestrian Detection](https://mlsanity.com/paper/2103.10091/) (2021)
- [Finding a Needle in a Haystack: Tiny Flying Object Detection in 4K Videos using a Joint Detection-and-Tracking Approach](https://mlsanity.com/paper/2105.08253/) (2021)
- [Evolutionary Neural Cascade Search across Supernetworks](https://mlsanity.com/paper/2203.04011/) (2022)
- [Reinforcement Learning as a Parsimonious Alternative to Prediction Cascades: A Case Study on Image Segmentation](https://mlsanity.com/paper/2402.11760/) (2024)
- [RedunCut: Measurement-Driven Sampling and Accuracy Performance Modeling for Low-Cost Live Video Analytics](https://mlsanity.com/paper/2512.24386/) (2025)

## Abstract (arXiv)

The design of complexity-aware cascaded detectors, combining features of very different complexities, is considered. A new cascade design procedure is introduced, by formulating cascade learning as the Lagrangian optimization of a risk that accounts for both accuracy and complexity. A boosting algorithm, denoted as complexity aware cascade training (CompACT), is then derived to solve this optimization. CompACT cascades are shown to seek an optimal trade-off between accuracy and complexity by pushing features of higher complexity to the later cascade stages, where only a few difficult candidate patches remain to be classified. This enables the use of features of vastly different complexities in a single detector. In result, the feature pool can be expanded to features previously impractical for cascade design, such as the responses of a deep convolutional neural network (CNN). This is demonstrated through the design of a pedestrian detector with a pool of features whose complexities span orders of magnitude. The resulting cascade generalizes the combination of a CNN with an object proposal mechanism: rather than a pre-processing stage, CompACT cascades seamlessly integrate CNNs in their stages. This enables state of the art performance on the Caltech and KITTI datasets, at fairly fast speeds.