# Automatically Leveraging MapReduce Frameworks for Data-Intensive Applications

arXiv: https://arxiv.org/abs/1801.09802 · HTML: https://mlsanity.com/paper/1801.09802/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jan 2018.

## Built on by (most cited first, 3 of 3)

- [ADELT: Transpilation Between Deep Learning Frameworks](https://mlsanity.com/paper/2303.03593/) (2023)
- [Report of the Workshop on Program Synthesis for Scientific Computing](https://mlsanity.com/paper/2102.01687/) (2021)
- [mlirSynth: Automatic, Retargetable Program Raising in Multi-Level IR using Program Synthesis](https://mlsanity.com/paper/2310.04196/) (2023)

## Abstract (arXiv)

MapReduce is a popular programming paradigm for developing large-scale, data-intensive computation. Many frameworks that implement this paradigm have recently been developed. To leverage these frameworks, however, developers must become familiar with their APIs and rewrite existing code. Casper is a new tool that automatically translates sequential Java programs into the MapReduce paradigm. Casper identifies potential code fragments to rewrite and translates them in two steps: (1) Casper uses program synthesis to search for a program summary (i.e., a functional specification) of each code fragment. The summary is expressed using a high-level intermediate language resembling the MapReduce paradigm and verified to be semantically equivalent to the original using a theorem prover. (2) Casper generates executable code from the summary, using either the Hadoop, Spark, or Flink API. We evaluated Casper by automatically converting real-world, sequential Java benchmarks to MapReduce. The resulting benchmarks perform up to 48.2x faster compared to the original.