# Worst-Case Control and Learning Using Partial Observations Over an Infinite Time-Horizon

arXiv: https://arxiv.org/abs/2303.16321 · HTML: https://mlsanity.com/paper/2303.16321/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 31 references that are in the map.
- First posted on arXiv 28 Mar 2023.

## Builds on (most cited first, 10 of 10)

- [Reducing the Hausdorff Distance in Medical Image Segmentation with Convolutional Neural Networks](https://mlsanity.com/paper/1904.10030/) (2019)
- [Approximate information state for approximate planning and reinforcement learning in partially observed systems](https://mlsanity.com/paper/2010.08843/) (2020)
- [Separation of Learning and Control for Cyber-Physical Systems](https://mlsanity.com/paper/2107.06379/) (2021)
- [Social Media and Misleading Information in a Democracy: A Mechanism Design Approach](https://mlsanity.com/paper/2003.07192/) (2020)
- [Optimal Time Trajectory and Coordination for Connected and Automated Vehicles](https://mlsanity.com/paper/2003.12183/) (2020)
- [Approximate Information States for Worst-case Control of Uncertain Systems](https://mlsanity.com/paper/2203.15271/) (2022)
- [Approximate Information States for Worst-Case Control and Learning in Uncertain Systems](https://mlsanity.com/paper/2301.05089/) (2023)
- [On Decentralized Minimax Control with Nested Subsystems](https://mlsanity.com/paper/2109.06328/) (2021)
- [Decentralized Control of Two Agents with Nested Accessible Information](https://mlsanity.com/paper/2109.11648/) (2021)
- [On Robust Control of Partially Observed Uncertain Systems with Additive Costs](https://mlsanity.com/paper/2209.13787/) (2022)

## Built on by (most cited first, 3 of 3)

- [Agent-state based policies in POMDPs: Beyond belief-state MDPs](https://mlsanity.com/paper/2409.15703/) (2024)
- [A Q-learning Approach for Adherence-Aware Recommendations](https://mlsanity.com/paper/2309.06519/) (2023)
- [A Framework for Effective AI Recommendations in Cyber-Physical-Human Systems](https://mlsanity.com/paper/2403.05715/) (2024)

## Abstract (arXiv)

Safety-critical cyber-physical systems require control strategies whose worst-case performance is robust against adversarial disturbances and modeling uncertainties. In this paper, we present a framework for approximate control and learning in partially observed systems to minimize the worst-case discounted cost over an infinite time horizon. We model disturbances to the system as finite-valued uncertain variables with unknown probability distributions. For problems with known system dynamics, we construct a dynamic programming (DP) decomposition to compute the optimal control strategy. Our first contribution is to define information states that improve the computational tractability of this DP without loss of optimality. Then, we describe a simplification for a class of problems where the incurred cost is observable at each time instance. Our second contribution is defining an approximate information state that can be constructed or learned directly from observed data for problems with observable costs. We derive bounds on the performance loss of the resulting approximate control strategy and illustrate the effectiveness of our approach in partially observed decision-making problems with a numerical example.