mlsanityData to 7 Oct 2026

Approximate Information States for Worst-Case Control and Learning in Uncertain Systems

arXiv 2301.05089 ↗First posted 12 Jan 2023eess.SY cs.AI cs.SYcited by 3

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate discrete-time decision-making problems in uncertain systems with partially observed states. We consider a non-stochastic model, where uncontrolled disturbances acting on the system take values in bounded sets with unknown distributions. We present a general framework for decision-making in such problems by using the notion of the information state and approximate information state, and introduce conditions to identify an uncertain variable that can be used to compute an optimal strategy through a dynamic program (DP). Next, we relax these conditions and define approximate information states that can be learned from output data without knowledge of system dynamics. We use approximate information states to formulate a DP that yields a strategy with a bounded performance loss. Finally, we illustrate the application of our results in control and reinforcement learning using numerical examples.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 33 of its 104 references that are in the map. First posted on arXiv 12 Jan 2023.

Builds on

33 of 104 references.

References · 33

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  3. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  4. 2016Cooperative Inverse Reinforcement Learningcited by 298
  5. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 129
  6. 2019Reducing the Hausdorff Distance in Medical Image Segmentation with Convolutional Neural Networkscited by 116
  7. 2019Online Control with Adversarial Disturbancescited by 92
  8. 2020Approximate information state for approximate planning and reinforcement learning in partially observed systemscited by 67
  9. 2020An Empirical Study of Representation Learning for Reinforcement Learning in Healthcarecited by 34
  10. 2021Medical Dead-ends and Learning to Identify High-risk States and Treatmentscited by 28
  11. 2020Near Optimality of Finite Memory Feedback Policies in Partially Observed Markov Decision Processescited by 20
  12. 2020Non-Stochastic Control with Bandit Feedbackcited by 17
21 more
  1. 2021On Team Decision Problems with Nonclassical Information Structurescited by 10
  2. 2021Separation of Learning and Control for Cyber-Physical Systemscited by 8
  3. 2021Common Information based Approximate State Representations in Multi-Agent Reinforcement Learningcited by 8
  4. 2021Distributed Asynchronous Policy Iteration for Sequential Zero-Sum Games and Minimax Controlcited by 6
  5. 2019Control Theory Meets POMDPs: A Hybrid Systems Approachcited by 5
  6. 2020Social Media and Misleading Information in a Democracy: A Mechanism Design Approachcited by 5
  7. 2020Optimal Time Trajectory and Coordination for Connected and Automated Vehiclescited by 5
  8. 2022Approximate Information States for Worst-case Control of Uncertain Systemscited by 4
  9. 2022A Cooperative Optimal Control Framework for Connected and Automated Vehicles in Mixed Traffic Using Social Value Orientationcited by 4
  10. 2022On learning history based policies for controlling Markov decision processescited by 4
  11. 2020An Overview on Optimal Flockingcited by 3
  12. 2017Symbolic Optimal Controlcited by 2
  13. 2021A Dynamic Program for a Team of Two Agents with Nested Informationcited by 2
  14. 2021On Decentralized Minimax Control with Nested Subsystemscited by 2
  15. 2021Decentralized Control of Two Agents with Nested Accessible Informationcited by 2
  16. 2022On Robust Control of Partially Observed Uncertain Systems with Additive Costscited by 2
  17. 2015A Duality Framework for Stochastic Optimal Control of Complex Systemscited by 1
  18. 2018Structural Results for Decentralized Stochastic Control with a Word-of-Mouth Communicationcited by 1
  19. 2019Worst-case Guarantees for Remote Estimation of an Uncertain Sourcecited by 1
  20. 2020Decentralized multi-agent reinforcement learning with shared actionscited by 1
  21. 2020Rethinking the Mathematical Framework and Optimality of Set-Membership Filteringcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1.

3 papers.

  1. 2024Agent-state based policies in POMDPs: Beyond belief-state MDPscited by 9
  2. 2023Connected and Automated Vehicles in Mixed-Traffic: Learning Human Driver Behavior for Effective On-Ramp Mergingcited by 7
  3. 2023Worst-Case Control and Learning Using Partial Observations Over an Infinite Time-Horizoncited by 3