# Active Perception with Initial-State Uncertainty: A Policy Gradient Method

arXiv: https://arxiv.org/abs/2409.16439 · HTML: https://mlsanity.com/paper/2409.16439/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 6 of its 16 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 6 of 6)

- [Revisiting Active Perception](https://mlsanity.com/paper/1603.02729/) (2016)
- [Smoother Entropy for Active State Trajectory Estimation and Obfuscation in POMDPs](https://mlsanity.com/paper/2108.10227/) (2021)
- [Multi-agent active perception with prediction rewards](https://mlsanity.com/paper/2010.11835/) (2020)
- [Entropy Maximization for Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2105.07490/) (2021)
- [Active Perception in Adversarial Scenarios using Maximum Entropy Deep Reinforcement Learning](https://mlsanity.com/paper/1902.05644/) (2019)
- [Information-Theoretic Opacity-Enforcement in Markov Decision Processes](https://mlsanity.com/paper/2405.00157/) (2024)

## Abstract (arXiv)

This paper studies the synthesis of an active perception policy that maximizes the information leakage of the initial state in a stochastic system modeled as a hidden Markov model (HMM). Specifically, the emission function of the HMM is controllable with a set of perception or sensor query actions. Given the goal is to infer the initial state from partial observations in the HMM, we use Shannon conditional entropy as the planning objective and develop a novel policy gradient method with convergence guarantees. By leveraging a variant of observable operators in HMMs, we prove several important properties of the gradient of the conditional entropy with respect to the policy parameters, which allow efficient computation of the policy gradient and stable and fast convergence. We demonstrate the effectiveness of our solution by applying it to an inference problem in a stochastic grid world environment.