# Covert Planning against Imperfect Observers

arXiv: https://arxiv.org/abs/2310.16791 · HTML: https://mlsanity.com/paper/2310.16791/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 26 references that are in the map.
- First posted on arXiv 25 Oct 2023.

## Builds on (most cited first, 8 of 8)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Near-Optimal Algorithms for Minimax Optimization](https://mlsanity.com/paper/2002.02417/) (2020)
- [Efficient Algorithms for Smooth Minimax Optimization](https://mlsanity.com/paper/1907.01543/) (2019)
- [Stochastic optimization with decision-dependent distributions](https://mlsanity.com/paper/2011.11173/) (2020)
- [Deceptive Reinforcement Learning for Privacy-Preserving Planning](https://mlsanity.com/paper/2102.03022/) (2021)
- [Deceptive Decision-Making Under Uncertainty](https://mlsanity.com/paper/2109.06740/) (2021)
- [Deceptive Reinforcement Learning in Model-Free Domains](https://mlsanity.com/paper/2303.10838/) (2023)
- [Deception in Supervisory Control](https://mlsanity.com/paper/1902.00590/) (2019)

## Built on by (most cited first, 2 of 2)

- [Information-Theoretic Opacity-Enforcement in Markov Decision Processes](https://mlsanity.com/paper/2405.00157/) (2024)
- [Optimal sensor deception in stochastic environments with partial observability to mislead a robot to a decoy goal](https://mlsanity.com/paper/2503.05972/) (2025)

## Abstract (arXiv)

Covert planning refers to a class of constrained planning problems where an agent aims to accomplish a task with minimal information leaked to a passive observer to avoid detection. However, existing methods of covert planning often consider deterministic environments or do not exploit the observer's imperfect information. This paper studies how covert planning can leverage the coupling of stochastic dynamics and the observer's imperfect observation to achieve optimal task performance without being detected. Specifically, we employ a Markov decision process to model the interaction between the agent and its stochastic environment, and a partial observation function to capture the leaked information to a passive observer. Assuming the observer employs hypothesis testing to detect if the observation deviates from a nominal policy, the covert planning agent aims to maximize the total discounted reward while keeping the probability of being detected as an adversary below a given threshold. We prove that finite-memory policies are more powerful than Markovian policies in covert planning. Then, we develop a primal-dual proximal policy gradient method with a two-time-scale update to compute a (locally) optimal covert policy. We demonstrate the effectiveness of our methods using a stochastic gridworld example. Our experimental results illustrate that the proposed method computes a policy that maximizes the adversary's expected reward without violating the detection constraint, and empirically demonstrates how the environmental noises can influence the performance of the covert policies.