mlsanityData to 7 Oct 2026

Adaptive Agent Design

arXiv 2609.12486 ↗First posted 11 Sep 2026eess.SY cs.AI cs.GTcited by 0

Not on the map. Why?

Abstract (arXiv)

We consider an agent acting against a general non-Markovian environment. The agent maintains its agent states, but is free to choose a transition kernel across those states and optimize its state-feedback control policies. We study the bi-level agent design problem that optimizes the transition kernel and the policy it induces, given said kernel with offline data of observations and actions obtained via a behavioral policy. For general environments, we show that a soft Q-learning algorithm converges almost surely to the fixed point of a soft Bellman equation defined by the stationary averages that the behavioral policy and the chosen kernel induce, and we delineate what separates the resulting policy from an optimal one. In partially observed Markov decision problems, we analyze convergence properties of parametrized transition kernel design via zero-th order and Bayesian optimization techniques.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 21 references that are in the map. First posted on arXiv 11 Sep 2026.

Builds on

13 of 21 references.

References · 13

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  3. 2013Optimal rates for zero-order convex optimization: the power of two function evaluationscited by 190
  4. 2020Approximate information state for approximate planning and reinforcement learning in partially observed systemscited by 67
  5. 2021Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent Statescited by 42
  6. 2021Convergence of Finite Memory Q-Learning for POMDPs and Near Optimality of Learned Policies under Filter Stabilitycited by 20
  7. 2020Information State Embedding in Partially Observable Cooperative Multi-Agent Reinforcement Learningcited by 13
  8. 2022Reinforcement Learning in Non-Markovian Environmentscited by 10
  9. 2024Agent-state based policies in POMDPs: Beyond belief-state MDPscited by 9
  10. 2025Scalable Policy-Based RL Algorithms for POMDPscited by 4
  11. 2023Value of Information in Games with Multiple Strategic Information Providerscited by 3
  12. 2024Learning How to Strategically Disclose Informationcited by 1
1 more
  1. 2025Harnessing Information in Incentive Designcited by 1

Built on by

Not cited yet.