# Least Inferable Policies for Markov Decision Processes

arXiv: https://arxiv.org/abs/1809.06482 · HTML: https://mlsanity.com/paper/1809.06482/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2018.

## Built on by (most cited first, 4 of 4)

- [Deceptive Sequential Decision-Making via Regularized Policy Optimization](https://mlsanity.com/paper/2501.18803/) (2025)
- [When Should a Leader Act Suboptimally? The Role of Inferability in Repeated Stackelberg Games](https://mlsanity.com/paper/2310.00468/) (2023)
- [Masking Radar Cognition under Adversarial Surveillance: A Distributional Privacy Framework](https://mlsanity.com/paper/2609.07428/) (2026)
- [Embedding Information in Policy Randomness: Distribution-Preserving Implicit Communication in MDPs](https://mlsanity.com/paper/2610.15976/) (2026)

## Abstract (arXiv)

In a variety of applications, an agent's success depends on the knowledge that an adversarial observer has or can gather about the agent's decisions. It is therefore desirable for the agent to achieve a task while reducing the ability of an observer to infer the agent's policy. We consider the task of the agent as a reachability problem in a Markov decision process and study the synthesis of policies that minimize the observer's ability to infer the transition probabilities of the agent between the states of the Markov decision process. We introduce a metric that is based on the Fisher information as a proxy for the information leaked to the observer and using this metric formulate a problem that minimizes expected total information subject to the reachability constraint. We proceed to solve the problem using convex optimization methods. To verify the proposed method, we analyze the relationship between the expected total information and the estimation error of the observer, and show that, for a particular class of Markov decision processes, these two values are inversely proportional.