Controlled Martingale Problems And Their Markov Mimics

In this article we prove under suitable assumptions that the marginals of any solution to a relaxed controlled martingale problem on a Polish space E can be mimicked by a Markovian solution of a Markov-relaxed controlled martingale problem. We also show how such `Markov mimics' can be obtained by relative entropy minimisation. We provide many examples where the above results can be applied.