Glivenko-Cantelli for f-divergence

We extend the celebrated Glivenko-Cantelli theorem, sometimes called the fundamental theorem of statistics, from its standard setting of total variation distance to all f-divergences. A key obstacle in this endeavor is to define f-divergence on a subcollection of a σ-algebra that forms a π-system but not a σ-subalgebra. This is a side contribution of our work. We will show that this notion of f-divergence on the π-system of rays preserves nearly all known properties of standard f-divergence, yields a novel integral representation of the Kolmogorov-Smirnov distance, and has a Glivenko-Cantelli theorem. We will also discuss the prospects of a Vapnik-Chervonenkis theory for f-divergence.