mlsanityData to 7 Oct 2026

Partially Observed Markov Decision Processes. Problem Sets and Internet Supplement

arXiv 1604.08127 ↗First posted 27 Apr 2016math.OCcited by 34

Not on the map. Why?

Abstract (arXiv)

This document is an internet supplement to my book "Partially Observed Markov Decision Processes - From Filtering to Controlled Sensing" published by Cambridge University Press in 2016. This internet supplement contains exercises, examples and case studies. The material appears in this internet supplement (instead of the book) so that it can be updated. This document will evolve over time and further discussion and examples will be added. This internet supplement document is work in progress and will be updated periodically. I welcome constructive comments from readers of the book and this internet supplement.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Apr 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 5; 2022: 4; 2023: 3; 2024: 6; 2025: 10; 2026: 4.

34 papers.

Sort:

By area:

  1. 2021Convergence of Finite Memory Q-Learning for POMDPs and Near Optimality of Learned Policies under Filter Stabilitycited by 20
  2. 2021Intrusion Prevention through Optimal Stoppingcited by 18
  3. 2024Conjectural Online Learning with First-order Beliefs in Asymmetric Information Stochastic Gamescited by 14
  4. 2019Identifying Cognitive Radars -- Inverse Reinforcement Learning using Revealed Preferencescited by 13
  5. 2023Learning Near-Optimal Intrusion Responses Against Dynamic Attackerscited by 13
  6. 2024Automated Security Response through Online Learning with Adaptive Conjecturescited by 13
  7. 2024Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Searchcited by 12
  8. 2021Learning Intrusion Prevention Policies through Optimal Stoppingcited by 11
  9. 2023Scalable Learning of Intrusion Responses through Recursive Decompositioncited by 8
  10. 2024Intrusion Tolerance for Networked Systems through Two-Level Feedback Controlcited by 8
  11. 2022Learning Security Strategies through Game Play and Optimal Stoppingcited by 5
  12. 2025Online Incident Response Planning under Model Misspecification through Bayesian Learning and Belief Quantizationcited by 5
22 more
  1. 2025Scalable Policy-Based RL Algorithms for POMDPscited by 4
  2. 2025Data-Assimilated Model-Based Reinforcement Learning for Partially Observed Chaotic Flowscited by 3
  3. 2025Learning Optimal Defender Strategies for CAGE-2 using a POMDP Modelcited by 3
  4. 2022Geometric Learning of Hidden Markov Models via a Method of Moments Algorithmcited by 2
  5. 2022Distributed Bayesian Learning of Dynamic Statescited by 2
  6. 2025Reinforcement Learning in Switching Non-Stationary Markov Decision Processes: Algorithms and Convergence Analysiscited by 2
  7. 2025CLT and Edgeworth Expansion for m-out-of-n Bootstrap Estimators of The Studentized Mediancited by 2
  8. 2021A Teacher-Student Markov Decision Process-based Framework for Online Correctional Learningcited by 1
  9. 2022How can a Radar Mask its Cognition?cited by 1
  10. 2023Policy Evaluation in Decentralized POMDPs with Belief Sharingcited by 1
  11. 2024Solving Truly Massive Budgeted Monotonic POMDPs with Oracle-Guided Meta-Reinforcement Learningcited by 1
  12. 2024Efficient Learning of POMDPs with Known Observation Model in Average-Reward Settingcited by 1
  13. 2025Data-assimilated model-informed reinforcement learningcited by 1
  14. 2026Knowing When to Ask for Help: Bayesian Self-Escalation in Hierarchical LLM Agentscited by 1
  15. 2020Langevin Dynamics for Adaptive Inverse Reinforcement Learning of Stochastic Gradient Algorithmscited by 0
  16. 2021Cell Multi-Bernoulli (Cell-MB) Sensor Control for Multi-object Search-While-Tracking (SWT)cited by 0
  17. 2025Achieving widetildeO(√T) Regret in Average-Reward POMDPs with Known Observation Modelscited by 0
  18. 2025Inferring Group Intent as a Cooperative Game. An NLP-based Framework for Trajectory Analysiscited by 0
  19. 2025Information-directed sampling for bandits: a primercited by 0
  20. 2026Prediction and Empowerment: A Theory of Agency through Bridge Interfacescited by 0
  21. 2026Optimal Stopping of Self-Refining Foundation Modelscited by 0
  22. 2026Learning Intrusion Response Strategies for OT Systemscited by 0