# Performance evaluation of job schedulers on Hadoop YARN

arXiv: https://arxiv.org/abs/1808.08289 · HTML: https://mlsanity.com/paper/1808.08289/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Aug 2018.

## Built on by (most cited first, 3 of 3)

- [How Far Should We Look Back to Achieve Effective Real-Time Time-Series Anomaly Detection?](https://mlsanity.com/paper/2102.06560/) (2021)
- [SALAD: Self-Adaptive Lightweight Anomaly Detection for Real-time Recurrent Time Series](https://mlsanity.com/paper/2104.09968/) (2021)
- [DistTune: Distributed Fine-Grained Adaptive Traffic Speed Prediction for Growing Transportation Networks](https://mlsanity.com/paper/2105.09421/) (2021)

## Abstract (arXiv)

To solve the limitation of Hadoop on scalability, resource sharing, and application support, the open-source community proposes the next generation of Hadoop's compute platform called Yet Another Resource Negotiator (YARN) by separating resource management functions from the programming model. This separation enables various application types to run on YARN in parallel. To achieve fair resource sharing and high resource utilization, YARN provides the capacity scheduler and the fair scheduler. However, the performance impacts of the two schedulers are not clear when mixed applications run on a YARN cluster. Therefore, in this paper, we study four scheduling-policy combinations (SPCs for short) derived from the two schedulers and then evaluate the four SPCs in extensive scenarios, which consider not only four application types, but also three different queue structures for organizing applications. The experimental results enable YARN managers to comprehend the influences of different SPCs and different queue structures on mixed applications. The results also help them to select a proper SPC and an appropriate queue structure to achieve better application execution performance.