# Markov Chain Neural Networks

arXiv: https://arxiv.org/abs/1805.00784 · HTML: https://mlsanity.com/paper/1805.00784/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2018.

## Built on by (most cited first, 6 of 6)

- [Constructing Neural Network-Based Models for Simulating Dynamical Systems](https://mlsanity.com/paper/2111.01495/) (2021)
- [MC-LSTM: Mass-Conserving LSTM](https://mlsanity.com/paper/2101.05186/) (2021)
- [Multi-Timescale Ensemble Q-learning for Markov Decision Process Policy Optimization](https://mlsanity.com/paper/2402.05476/) (2024)
- [On the Stochastic Stability of Deep Markov Models](https://mlsanity.com/paper/2111.04601/) (2021)
- [Neural calibration of hidden inhomogeneous Markov chains -- Information decompression in life insurance](https://mlsanity.com/paper/2201.02397/) (2022)
- [Learning Time-Inhomogeneous Markov Dynamics in Financial Time Series via Neural Parameterization](https://mlsanity.com/paper/2605.04690/) (2026)

## Abstract (arXiv)

In this work we present a modified neural network model which is capable to simulate Markov Chains. We show how to express and train such a network, how to ensure given statistical properties reflected in the training data and we demonstrate several applications where the network produces non-deterministic outcomes. One example is a random walker model, e.g. useful for simulation of Brownian motions or a natural Tic-Tac-Toe network which ensures non-deterministic game behavior.