# PlinyCompute: A Platform for High-Performance, Distributed, Data-Intensive Tool Development

arXiv: https://arxiv.org/abs/1711.05573 · HTML: https://mlsanity.com/paper/1711.05573/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Nov 2017.

## Built on by (most cited first, 5 of 5)

- [A Comparison of Decision Forest Inference Platforms from A Database Perspective](https://mlsanity.com/paper/2302.04430/) (2023)
- [Serving Deep Learning Model in Relational Databases](https://mlsanity.com/paper/2310.04696/) (2023)
- [Auto-Differentiation of Relational Computations for Very Large Scale Machine Learning](https://mlsanity.com/paper/2306.00088/) (2023)
- [InferF: Declarative Factorization of AI/ML Inferences over Joins](https://mlsanity.com/paper/2511.20489/) (2025)
- [Automated Tensor-Relational Decomposition for Large-Scale Sparse Tensor Computation](https://mlsanity.com/paper/2603.08957/) (2026)

## Abstract (arXiv)

This paper describes PlinyCompute, a system for development of high-performance, data-intensive, distributed computing tools and libraries. In the large, PlinyCompute presents the programmer with a very high-level, declarative interface, relying on automatic, relational-database style optimization to figure out how to stage distributed computations. However, in the small, PlinyCompute presents the capable systems programmer with a persistent object data model and API (the "PC object model") and associated memory management system that has been designed from the ground-up for high performance, distributed, data-intensive computing. This contrasts with most other Big Data systems, which are constructed on top of the Java Virtual Machine (JVM), and hence must at least partially cede performance-critical concerns such as memory management (including layout and de/allocation) and virtual method/function dispatch to the JVM. This hybrid approach---declarative in the large, trusting the programmer's ability to utilize PC object model efficiently in the small---results in a system that is ideal for the development of reusable, data-intensive tools and libraries. Through extensive benchmarking, we show that implementing complex objects manipulation and non-trivial, library-style computations on top of PlinyCompute can result in a speedup of 2x to more than 50x or more compared to equivalent implementations on Spark.