# Testing for the Markov Property in Time Series via Deep Conditional Generative Learning

arXiv: https://arxiv.org/abs/2305.19244 · HTML: https://mlsanity.com/paper/2305.19244/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 70 references that are in the map.
- First posted on arXiv 30 May 2023.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [TensorFlow Distributions](https://mlsanity.com/paper/1711.10604/) (2017)
- [Attribute2Image: Conditional Image Generation from Visual Attributes](https://mlsanity.com/paper/1512.00570/) (2015)
- [How Well Generative Adversarial Networks Learn Distributions](https://mlsanity.com/paper/1811.03179/) (2018)
- [Statistical Inference of the Value Function for Reinforcement Learning in Infinite Horizon Settings](https://mlsanity.com/paper/2001.04515/) (2020)
- [Deep Neural Networks for Estimation and Inference](https://mlsanity.com/paper/1809.09953/) (2018)
- [Deep Neural Networks Learn Non-Smooth Functions Effectively](https://mlsanity.com/paper/1802.04474/) (2018)
- [Conditional Density Estimation with Neural Networks: Best Practices and Benchmarks](https://mlsanity.com/paper/1903.00954/) (2019)
- [Does the Markov Decision Process Fit the Data: Testing for the Markov Property in Sequential Decision Making](https://mlsanity.com/paper/2002.01751/) (2020)
- [On Computation and Generalization of Generative Adversarial Imitation Learning](https://mlsanity.com/paper/2001.02792/) (2020)
- [Bottleneck Conditional Density Estimation](https://mlsanity.com/paper/1611.08568/) (2016)
- [Nuclei Detection Using Mixture Density Networks](https://mlsanity.com/paper/1808.08279/) (2018)
- [Principal component analysis for second-order stationary vector time series](https://mlsanity.com/paper/1410.2323/) (2014)
- [Parametric-Rate Inference for One-Sided Differentiable Parameters](https://mlsanity.com/paper/1605.07657/) (2016)

## Built on by (most cited first, 9 of 9)

- [Doubly Robust Conditional Independence Testing with Generative Neural Networks](https://mlsanity.com/paper/2407.17694/) (2024)
- [Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experiments](https://mlsanity.com/paper/2403.17285/) (2024)
- [Model Free Prediction with Uncertainty Assessment](https://mlsanity.com/paper/2405.12684/) (2024)
- [Statistical Inference in Reinforcement Learning: A Selective Survey](https://mlsanity.com/paper/2502.16195/) (2025)
- [A Conditional Distribution Equality Testing Framework using Deep Generative Learning](https://mlsanity.com/paper/2509.17729/) (2025)
- [Time Series Generative Learning with Application to Brain Imaging Analysis](https://mlsanity.com/paper/2407.14003/) (2024)
- [Capturing reduced-order quantum many-body dynamics out of equilibrium via neural ordinary differential equations](https://mlsanity.com/paper/2512.13913/) (2025)
- [From Causal Graphs to Markovian States: Multi-Order State Exposure for Deep RL](https://mlsanity.com/paper/2605.07057/) (2026)
- [Generative sequence modeling for infinite memory processes via predictive states](https://mlsanity.com/paper/2609.38524/) (2026)

## Abstract (arXiv)

The Markov property is widely imposed in analysis of time series data. Correspondingly, testing the Markov property, and relatedly, inferring the order of a Markov model, are of paramount importance. In this article, we propose a nonparametric test for the Markov property in high-dimensional time series via deep conditional generative learning. We also apply the test sequentially to determine the order of the Markov model. We show that the test controls the type-I error asymptotically, and has the power approaching one. Our proposal makes novel contributions in several ways. We utilize and extend state-of-the-art deep generative learning to estimate the conditional density functions, and establish a sharp upper bound on the approximation error of the estimators. We derive a doubly robust test statistic, which employs a nonparametric estimation but achieves a parametric convergence rate. We further adopt sample splitting and cross-fitting to minimize the conditions required to ensure the consistency of the test. We demonstrate the efficacy of the test through both simulations and the three data applications.