# NeRCC: Nested-Regression Coded Computing for Resilient Distributed Prediction Serving Systems

arXiv: https://arxiv.org/abs/2402.04377 · HTML: https://mlsanity.com/paper/2402.04377/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 6 Feb 2024.

## Builds on (most cited first, 13 of 13)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021)
- [Speeding Up Distributed Machine Learning Using Codes](https://mlsanity.com/paper/1512.02673/) (2015)
- [Lagrange Coded Computing: Optimal Design for Resiliency, Security and Privacy](https://mlsanity.com/paper/1806.00939/) (2018)
- [ErasureHead: Distributed Gradient Descent without Delays Using Approximate Gradient Coding](https://mlsanity.com/paper/1901.09671/) (2019)
- [A Scalable Approach for Privacy-Preserving Collaborative Machine Learning](https://mlsanity.com/paper/2011.01963/) (2020)
- [Berrut Approximated Coded Computing: Straggler Resistance Beyond Polynomial Computing](https://mlsanity.com/paper/2009.08327/) (2020)
- [Fundamental Limits of Approximate Gradient Coding](https://mlsanity.com/paper/1901.08166/) (2019)
- [When Do Redundant Requests Reduce Latency ?](https://mlsanity.com/paper/1311.2851/) (2013)
- [Learning a Code: Machine Learning for Approximate Non-Linear Coded Computation](https://mlsanity.com/paper/1806.01259/) (2018)
- [Parity Models: A General Framework for Coding-Based Resilience in ML Inference](https://mlsanity.com/paper/1905.00863/) (2019)
- [Slack Squeeze Coded Computing for Adaptive Straggler Mitigation](https://mlsanity.com/paper/1904.07098/) (2019)
- [Coded-InvNet for Resilient Prediction Serving Systems](https://mlsanity.com/paper/2106.06445/) (2021)

## Abstract (arXiv)

Resilience against stragglers is a critical element of prediction serving systems, tasked with executing inferences on input data for a pre-trained machine-learning model. In this paper, we propose NeRCC, as a general straggler-resistant framework for approximate coded computing. NeRCC includes three layers: (1) encoding regression and sampling, which generates coded data points, as a combination of original data points, (2) computing, in which a cluster of workers run inference on the coded data points, (3) decoding regression and sampling, which approximately recovers the predictions of the original data points from the available predictions on the coded data points. We argue that the overall objective of the framework reveals an underlying interconnection between two regression models in the encoding and decoding layers. We propose a solution to the nested regressions problem by summarizing their dependence on two regularization terms that are jointly optimized. Our extensive experiments on different datasets and various machine learning models, including LeNet5, RepVGG, and Vision Transformer (ViT), demonstrate that NeRCC accurately approximates the original predictions in a wide range of stragglers, outperforming the state-of-the-art by up to 23%.