# Approximate Information States for Worst-Case Control and Learning in Uncertain Systems

arXiv: https://arxiv.org/abs/2301.05089 · HTML: https://mlsanity.com/paper/2301.05089/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 104 references that are in the map.
- First posted on arXiv 12 Jan 2023.

## Builds on (most cited first, 30 of 33)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [Reducing the Hausdorff Distance in Medical Image Segmentation with Convolutional Neural Networks](https://mlsanity.com/paper/1904.10030/) (2019)
- [Online Control with Adversarial Disturbances](https://mlsanity.com/paper/1902.08721/) (2019)
- [Approximate information state for approximate planning and reinforcement learning in partially observed systems](https://mlsanity.com/paper/2010.08843/) (2020)
- [An Empirical Study of Representation Learning for Reinforcement Learning in Healthcare](https://mlsanity.com/paper/2011.11235/) (2020)
- [Medical Dead-ends and Learning to Identify High-risk States and Treatments](https://mlsanity.com/paper/2110.04186/) (2021)
- [Near Optimality of Finite Memory Feedback Policies in Partially Observed Markov Decision Processes](https://mlsanity.com/paper/2010.07452/) (2020)
- [Non-Stochastic Control with Bandit Feedback](https://mlsanity.com/paper/2008.05523/) (2020)
- [On Team Decision Problems with Nonclassical Information Structures](https://mlsanity.com/paper/2101.10992/) (2021)
- [Separation of Learning and Control for Cyber-Physical Systems](https://mlsanity.com/paper/2107.06379/) (2021)
- [Common Information based Approximate State Representations in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2110.12603/) (2021)
- [Distributed Asynchronous Policy Iteration for Sequential Zero-Sum Games and Minimax Control](https://mlsanity.com/paper/2107.10406/) (2021)
- [Control Theory Meets POMDPs: A Hybrid Systems Approach](https://mlsanity.com/paper/1905.08095/) (2019)
- [Social Media and Misleading Information in a Democracy: A Mechanism Design Approach](https://mlsanity.com/paper/2003.07192/) (2020)
- [Optimal Time Trajectory and Coordination for Connected and Automated Vehicles](https://mlsanity.com/paper/2003.12183/) (2020)
- [On learning history based policies for controlling Markov decision processes](https://mlsanity.com/paper/2211.03011/) (2022)
- [Approximate Information States for Worst-case Control of Uncertain Systems](https://mlsanity.com/paper/2203.15271/) (2022)
- [A Cooperative Optimal Control Framework for Connected and Automated Vehicles in Mixed Traffic Using Social Value Orientation](https://mlsanity.com/paper/2203.17106/) (2022)
- [An Overview on Optimal Flocking](https://mlsanity.com/paper/2009.14279/) (2020)
- [Symbolic Optimal Control](https://mlsanity.com/paper/1709.07333/) (2017)
- [A Dynamic Program for a Team of Two Agents with Nested Information](https://mlsanity.com/paper/2103.10028/) (2021)
- [On Decentralized Minimax Control with Nested Subsystems](https://mlsanity.com/paper/2109.06328/) (2021)
- [Decentralized Control of Two Agents with Nested Accessible Information](https://mlsanity.com/paper/2109.11648/) (2021)
- [On Robust Control of Partially Observed Uncertain Systems with Additive Costs](https://mlsanity.com/paper/2209.13787/) (2022)
- [A Duality Framework for Stochastic Optimal Control of Complex Systems](https://mlsanity.com/paper/1506.03417/) (2015)
- [Structural Results for Decentralized Stochastic Control with a Word-of-Mouth Communication](https://mlsanity.com/paper/1809.08712/) (2018)

## Built on by (most cited first, 3 of 3)

- [Agent-state based policies in POMDPs: Beyond belief-state MDPs](https://mlsanity.com/paper/2409.15703/) (2024)
- [Connected and Automated Vehicles in Mixed-Traffic: Learning Human Driver Behavior for Effective On-Ramp Merging](https://mlsanity.com/paper/2304.00397/) (2023)
- [Worst-Case Control and Learning Using Partial Observations Over an Infinite Time-Horizon](https://mlsanity.com/paper/2303.16321/) (2023)

## Abstract (arXiv)

In this paper, we investigate discrete-time decision-making problems in uncertain systems with partially observed states. We consider a non-stochastic model, where uncontrolled disturbances acting on the system take values in bounded sets with unknown distributions. We present a general framework for decision-making in such problems by using the notion of the information state and approximate information state, and introduce conditions to identify an uncertain variable that can be used to compute an optimal strategy through a dynamic program (DP). Next, we relax these conditions and define approximate information states that can be learned from output data without knowledge of system dynamics. We use approximate information states to formulate a DP that yields a strategy with a bounded performance loss. Finally, we illustrate the application of our results in control and reinforcement learning using numerical examples.