mlsanityData to 14 Oct 2026

Active Perception with Initial-State Uncertainty: A Policy Gradient Method

arXiv 2409.16439 ↗First posted 24 Sep 2024eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

This paper studies the synthesis of an active perception policy that maximizes the information leakage of the initial state in a stochastic system modeled as a hidden Markov model (HMM). Specifically, the emission function of the HMM is controllable with a set of perception or sensor query actions. Given the goal is to infer the initial state from partial observations in the HMM, we use Shannon conditional entropy as the planning objective and develop a novel policy gradient method with convergence guarantees. By leveraging a variant of observable operators in HMMs, we prove several important properties of the gradient of the conditional entropy with respect to the policy parameters, which allow efficient computation of the policy gradient and stable and fast convergence. We demonstrate the effectiveness of our solution by applying it to an inference problem in a stochastic grid world environment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 6 of its 16 references that are in the map. First posted on arXiv 24 Sep 2024.

Builds on

6 of 16 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Information-Theoretic Opacity-Enforcement in Markov Decision Processes30 Apr

Other references · 5

  1. 2016Revisiting Active Perceptioncited by 94
  2. 2021Smoother Entropy for Active State Trajectory Estimation and Obfuscation in POMDPscited by 6
  3. 2020Multi-agent active perception with prediction rewardscited by 5
  4. 2021Entropy Maximization for Partially Observable Markov Decision Processescited by 4
  5. 2019Active Perception in Adversarial Scenarios using Maximum Entropy Deep Reinforcement Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Integrated Control and Active Perception in POMDPs for Temporal Logic Tasks and Information Acquisition ↗
  2. 2025Policy Gradient Methods for Information-Theoretic Opacity in Markov Decision Processes
  3. 2026Active Inference as a Convex Markov Decision Process
  4. 2025Apple: Toward General Active Perception via Reinforcement Learning
  5. 2025ACTIVE-o3: Empowering MLLMs with Active Perception via Pure Reinforcement Learning
  6. 2025Synthesis of Dynamic Masks for Information-Theoretic Opacity in Stochastic Systems
  7. 2026Policy Gradient Methods for Non-Markovian Reinforcement Learning
  8. 2025Real-World Reinforcement Learning of Active Perception Behaviors