# Accelerating Deep Learning Classification with Error-controlled Approximate-key Caching

arXiv: https://arxiv.org/abs/2112.06671 · HTML: https://mlsanity.com/paper/2112.06671/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 52 references that are in the map.
- First posted on arXiv 13 Dec 2021.

## Builds on (most cited first, 16 of 16)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Clipper: A Low-Latency Online Prediction Serving System](https://mlsanity.com/paper/1612.03079/) (2016)
- [SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud](https://mlsanity.com/paper/2008.06402/) (2020)
- [DeepCache: Principled Cache for Mobile Deep Vision](https://mlsanity.com/paper/1712.01670/) (2017)
- [Benchmarking TPU, GPU, and CPU Platforms for Deep Learning](https://mlsanity.com/paper/1907.10701/) (2019)
- [Deep Learning and Traffic Classification: Lessons learned from a commercial-grade dataset with hundreds of encrypted and zero-day applications](https://mlsanity.com/paper/2104.03182/) (2021)
- [Why (and How) Networks Should Run Themselves](https://mlsanity.com/paper/1710.11583/) (2017)
- [LCNN: Lookup-based Convolutional Neural Network](https://mlsanity.com/paper/1611.06473/) (2016)
- [Accelerating Deep Learning Inference via Freezing](https://mlsanity.com/paper/2002.02645/) (2020)
- [The Missing Piece in Complex Analytics: Low Latency, Scalable Model Management and Serving with Velox](https://mlsanity.com/paper/1409.3809/) (2014)
- [Return of the Lernaean Hydra: Experimental Evaluation of Data Series Approximate Similarity Search](https://mlsanity.com/paper/2006.11459/) (2020)
- [A unified approach to the performance analysis of caching systems](https://mlsanity.com/paper/1307.6702/) (2013)
- [Similarity Caching: Theory and Algorithms](https://mlsanity.com/paper/1912.03888/) (2019)
- [Femto-Caching with Soft Cache Hits: Improving Performance through Recommendation and Delivery of Related Content](https://mlsanity.com/paper/1702.04943/) (2017)

## Built on by (most cited first, 2 of 2)

- [Landing AI on Networks: An equipment vendor viewpoint on Autonomous Driving Networks](https://mlsanity.com/paper/2205.08347/) (2022)
- [Serving Deep Learning Model in Relational Databases](https://mlsanity.com/paper/2310.04696/) (2023)

## Abstract (arXiv)

While Deep Learning (DL) technologies are a promising tool to solve networking problems that map to classification tasks, their computational complexity is still too high with respect to real-time traffic measurements requirements. To reduce the DL inference cost, we propose a novel caching paradigm, that we named approximate-key caching, which returns approximate results for lookups of selected input based on cached DL inference results. While approximate cache hits alleviate DL inference workload and increase the system throughput, they however introduce an approximation error. As such, we couple approximate-key caching with an error-correction principled algorithm, that we named auto-refresh. We analytically model our caching system performance for classic LRU and ideal caches, we perform a trace-driven evaluation of the expected performance, and we compare the benefits of our proposed approach with the state-of-the-art similarity caching -- testifying the practical interest of our proposal.