# Approximation Algorithms for Cascading Prediction Models

arXiv: https://arxiv.org/abs/1802.07697 · HTML: https://mlsanity.com/paper/1802.07697/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2018.

## Built on by (most cited first, 9 of 9)

- [When Does Confidence-Based Cascade Deferral Suffice?](https://mlsanity.com/paper/2307.02764/) (2023)
- [Lossless Acceleration for Seq2seq Generation with Aggressive Decoding](https://mlsanity.com/paper/2205.10350/) (2022)
- [How to Fix a Broken Confidence Estimator: Evaluating Post-hoc Methods for Selective Classification with Deep Neural Networks](https://mlsanity.com/paper/2305.15508/) (2023)
- [Agreement-Based Cascading for Efficient Inference](https://mlsanity.com/paper/2407.02348/) (2024)
- [Cascade-Aware Training of Language Models](https://mlsanity.com/paper/2406.00060/) (2024)
- [BabyBear: Cheap inference triage for expensive language models](https://mlsanity.com/paper/2205.11747/) (2022)
- [Cascadia: An Efficient Cascade Serving System for Large Language Models](https://mlsanity.com/paper/2506.04203/) (2025)
- [KLAS: Using Similarity to Stitch Neural Networks for Improved Accuracy-Efficiency Tradeoffs](https://mlsanity.com/paper/2605.29259/) (2026)
- [Evolutionary Neural Cascade Search across Supernetworks](https://mlsanity.com/paper/2203.04011/) (2022)

## Abstract (arXiv)

We present an approximation algorithm that takes a pool of pre-trained models as input and produces from it a cascaded model with similar accuracy but lower average-case cost. Applied to state-of-the-art ImageNet classification models, this yields up to a 2x reduction in floating point multiplications, and up to a 6x reduction in average-case memory I/O. The auto-generated cascades exhibit intuitive properties, such as using lower-resolution input for easier images and requiring higher prediction confidence when using a computationally cheaper model.