# Smoother Entropy for Active State Trajectory Estimation and Obfuscation in POMDPs

arXiv: https://arxiv.org/abs/2108.10227 · HTML: https://mlsanity.com/paper/2108.10227/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 72 references that are in the map.
- First posted on arXiv 19 Aug 2021.

## Builds on (most cited first, 21 of 21)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [MAGIC: Learning Macro-Actions for Online POMDP Planning](https://mlsanity.com/paper/2011.03813/) (2020)
- [An On-Line POMDP Solver for Continuous Observation Spaces](https://mlsanity.com/paper/2011.02076/) (2020)
- [Entropy Maximization for Markov Decision Processes Under Temporal Logic Constraints](https://mlsanity.com/paper/1807.03223/) (2018)
- [Policy Design for Active Sequential Hypothesis Testing using Deep Learning](https://mlsanity.com/paper/1810.04859/) (2018)
- [How to Calibrate your Adversary's Capabilities? Inverse Filtering for Counter-Autonomous Systems](https://mlsanity.com/paper/1905.07230/) (2019)
- [Information-based Active SLAM via Topological Feature Graphs](https://mlsanity.com/paper/1509.08155/) (2015)
- [Inverse Filtering for Hidden Markov Models with Applications to Counter-Adversarial Autonomous Systems](https://mlsanity.com/paper/2001.11809/) (2020)
- [Active Classification for POMDPs: a Kalman-like State Estimator](https://mlsanity.com/paper/1312.2039/) (2013)
- [Monte Carlo Information-Oriented Planning](https://mlsanity.com/paper/2103.11345/) (2021)
- [Real-Time Privacy-Preserving Data Release for Smart Meters](https://mlsanity.com/paper/1906.06427/) (2019)
- [How to Protect Your Privacy? A Framework for Counter-Adversarial Decision Making](https://mlsanity.com/paper/2004.04119/) (2020)
- [Unpredictable Planning Under Partial Observability](https://mlsanity.com/paper/1903.07665/) (2019)
- [Convex Stochastic Dominance in Bayesian Localization, Filtering and Controlled Sensing POMDPs](https://mlsanity.com/paper/1904.00287/) (2019)
- [Active Trajectory Estimation for Partially Observed Markov Decision Processes via Conditional Entropy](https://mlsanity.com/paper/2104.01545/) (2021)
- [Differentially Private State Estimation in Distribution Networks with Smart Meters](https://mlsanity.com/paper/1503.08490/) (2015)
- [Differentially Private Cloud-Based Multi-Agent Optimization with Constraints](https://mlsanity.com/paper/1708.08422/) (2017)
- [Detection-averse optimal and receding-horizon control for Markov decision processes](https://mlsanity.com/paper/1908.07691/) (2019)
- [On Privacy of Dynamical Systems: An Optimal Probabilistic Mapping Approach (Extended Preprint)](https://mlsanity.com/paper/1910.13559/) (2019)
- [Smoothing-Averse Control: Covertness and Privacy from Smoothers](https://mlsanity.com/paper/2103.12881/) (2021)

## Built on by (most cited first, 6 of 6)

- [Information-Theoretic Opacity-Enforcement in Markov Decision Processes](https://mlsanity.com/paper/2405.00157/) (2024)
- [Entropy-Regularized Partially Observed Markov Decision Processes](https://mlsanity.com/paper/2112.12255/) (2021)
- [Active Perception with Initial-State Uncertainty: A Policy Gradient Method](https://mlsanity.com/paper/2409.16439/) (2024)
- [Convergence of regularized agent-state-based Q-learning in POMDPs](https://mlsanity.com/paper/2508.21314/) (2025)
- [Policy Gradient Methods for Information-Theoretic Opacity in Markov Decision Processes](https://mlsanity.com/paper/2511.02704/) (2025)
- [Goal inference with Rao-Blackwellized Particle Filters](https://mlsanity.com/paper/2512.09269/) (2025)

## Abstract (arXiv)

We study the problem of controlling a partially observed Markov decision process (POMDP) to either aid or hinder the estimation of its state trajectory. We encode the estimation objectives via the smoother entropy, which is the conditional entropy of the state trajectory given measurements and controls. Consideration of the smoother entropy contrasts with previous approaches that instead resort to marginal (or instantaneous) state entropies due to tractability concerns. By establishing novel expressions for the smoother entropy in terms of the POMDP belief state, we show that both the problems of minimising and maximising the smoother entropy in POMDPs can surprisingly be reformulated as belief-state Markov decision processes with concave cost and value functions. The significance of these reformulations is that they render the smoother entropy a tractable optimisation objective, with structural properties amenable to the use of standard POMDP solution techniques for both active estimation and obfuscation. Simulations illustrate that optimisation of the smoother entropy leads to superior trajectory estimation and obfuscation compared to alternative approaches.