# A Survey of State Management in Big Data Processing Systems

arXiv: https://arxiv.org/abs/1702.01596 · HTML: https://mlsanity.com/paper/1702.01596/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2017.

## Built on by (most cited first, 2 of 2)

- [Detecting Rumours with Latency Guarantees using Massive Streaming Data](https://mlsanity.com/paper/2205.06580/) (2022)
- [DataSculpt: Crafting Data Landscapes for Long-Context LLMs through Multi-Objective Partitioning](https://mlsanity.com/paper/2409.00997/) (2024)

## Abstract (arXiv)

State management and its use in diverse applications varies widely across big data processing systems. This is evident in both the research literature and existing systems, such as Apache Flink, Apache Samza, Apache Spark, and Apache Storm. Given the pivotal role that state management plays in various use cases, in this survey, we present some of the most important uses of state as an enabler, discuss the alternative approaches used to handle and implement state, propose a taxonomy to capture the many facets of state management, and highlight new research directions. Our aim is to provide insight into disparate state management techniques, motivate others to pursue research in this area, and draw attention to some open problems.