mlsanityData to 7 Oct 2026

Pessimistic Iterative Planning with RNNs for Robust POMDPs

arXiv 2408.08770 ↗First posted 16 Aug 2024cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Robust POMDPs extend classical POMDPs to incorporate model uncertainty using so-called uncertainty sets on the transition and observation functions, effectively defining ranges of probabilities. Policies for robust POMDPs must be (1) memory-based to account for partial observability and (2) robust against model uncertainty to account for the worst-case probability instances from the uncertainty sets. To compute such robust memory-based policies, we propose the pessimistic iterative planning (PIP) framework, which alternates between (1) selecting pessimistic POMDPs via worst-case probability instances from the uncertainty sets, and (2) computing finite-state controllers (FSCs) for these pessimistic POMDPs. Within PIP, we propose the rFSCNet algorithm, which optimizes a recurrent neural network to compute the FSCs. The empirical evaluation shows that rFSCNet can compute better-performing robust policies than several baselines and a state-of-the-art robust POMDP solver.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 17 of its 81 references that are in the map. First posted on arXiv 16 Aug 2024.

Builds on

17 of 81 references.

References · 17

  1. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  3. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
  4. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
  5. 2020The Probabilistic Model Checker Stormcited by 64
  6. 2022Safe Reinforcement Learning via Shielding under Partial Observabilitycited by 49
  7. 2018Learning Finite State Representations of Recurrent Policy Networkscited by 37
  8. 2013Solving POMDPs by Searching the Space of Finite Policiescited by 25
  9. 2020Robust Finite-State Controllers for Uncertain POMDPscited by 23
  10. 2019Distributionally Robust Partially Observable Markov Decision Process with Moment-based Ambiguitycited by 15
  11. 2020Enforcing Almost-Sure Reachability in POMDPscited by 13
  12. 2014Optimal Cost Almost-sure Reachability in POMDPscited by 12
5 more
  1. 2022Recurrent networks, hidden states and beliefs in partially observable environmentscited by 11
  2. 2014Qualitative Analysis of POMDPs with Temporal Logic Specifications for Robotics Applicationscited by 8
  3. 2024Imprecise Probabilities Meet Partial Observability: Game Semantics for Robust POMDPscited by 7
  4. 2021The Stackelberg Equilibrium for One-sided Zero-sum Partially Observable Stochastic Gamescited by 6
  5. 2020Robust Policy Synthesis for Uncertain POMDPs via Convex Optimizationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 4.

7 papers.

  1. 2024Robust Markov Decision Processes: A Place Where AI and Formal Methods Meetcited by 18
  2. 2025Robust Finite-Memory Policy Gradients for Hidden-Model POMDPscited by 5
  3. 2025Multi-Environment POMDPs: Discrete Model Uncertainty Under Partial Observabilitycited by 2
  4. 2026Finite-State Controllers for (Hidden-Model) POMDPs using Deep Reinforcement Learningcited by 1
  5. 2026Robustness Analysis of POMDP Policies to Observation Perturbationscited by 0
  6. 2026Solving Robust POMDPs with Omega-regular Objectives via Partially Observable Stochastic Gamescited by 0
  7. 2026Distinguish or Homogenize: Last-Chance Policy Identification and Risk-Budgeted Recovery under Irreversible Resource Depletioncited by 0