On The Suitability of Differential Dataflow For Datalog Interpretation In Highly Dynamic Settings

In the domain of knowledge representation and reasoning within AI, datalog engines play an ever-increasingly crucial role. The crux of their operation lies in materialization: the evaluation of a data- log program and its incorporation into a database. This operation becomes complex and resource-intensive, especially when the data is highly dynamic, as it is common in distributed environments. Thus, incremental materialization, adjusting the computation to new data instead of restarting it, is the norm. However, handling the deletion of data is significantly more complicated than addition due to the cascading effects of what is being removed. Differential Dataflow offers a computational model that effectively addresses this, ensuring consistent performance for both data additions and deletions. In this paper, we delve into the efficiency of materializa- tion using three distinct datalog implementations: one based on a streamlined relational engine and two others that implement the same algorithm, but with one utilizing differential-dataflow, and another not. Our insights provide a roadmap for enhancing datalog- driven computations, particularly in dynamic data environments like the cloud.