mlsanityData to 7 Oct 2026

Coachable agents for interactive gameplay

arXiv 2607.00642 ↗First posted 1 Jul 2026cs.AI cs.LGcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Reinforcement learning has proven to be a valuable tool in the creation of advanced AI and robotic systems, contributing to everything from game playing to robotics to foundation models. Through trial-and-error, these AI systems typically learn one, near-optimal behavior to solve their tasks. However, there are many use cases in which one would like to assert some level of control, preferably in real time, over how the task is solved. We refer to these modifications of a core task as styles. We combine universal value function approximators (UVFAs) with carefully selected training scenarios, learning algorithms, and data augmentation to create a framework for coaching agents that exhibit styles in complex domains. We demonstrate the framework's application in the AAA video games Horizon Forbidden West and Gran Turismo, and in an open-source humanoid test domain. Despite the different nature of the domains -- car racing, stylized game combat, and humanoid walking -- each agent shows strong coherence to the style requests while still satisfying the main task in its domain. Importantly, the techniques outlined in this paper allow an end user to choose the final behavior at run time, giving them flexible control over the final executed performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 35 of its 63 references that are in the map. First posted on arXiv 1 Jul 2026.

Builds on

35 of 63 references.

References · 35

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  6. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  7. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  8. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  9. 2018DeepMind Control Suitecited by 935
  10. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  11. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  12. 2018Learning Dexterous In-Hand Manipulationcited by 643
23 more
  1. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  2. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  3. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  4. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 240
  5. 2017DeepStack: Expert-Level Artificial Intelligence in No-Limit Pokercited by 228
  6. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  7. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learningcited by 153
  8. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  9. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 136
  10. 2018Dynamic Weights in Multi-Objective Deep Reinforcement Learningcited by 96
  11. 2021A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learningcited by 82
  12. 2014Solving Large Imperfect Information Games Using CFR+cited by 78
  13. 2017Control of a Quadrotor with Reinforcement Learningcited by 75
  14. 2018Improving Regression Performance with Distributional Lossescited by 66
  15. 2024Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Modelscited by 65
  16. 2021Student of Games: A unified learning algorithm for both perfect and imperfect information gamescited by 22
  17. 2024A Comprehensive Survey of Reinforcement Learning: From Algorithms to Practical Challengescited by 13
  18. 2014Solving Games with Functional Regret Estimationcited by 12
  19. 2022Automated Play-Testing Through RL Based Human-Like Play-Styles Generationcited by 11
  20. 2025A Critical Assessment of Modern Generative Models' Ability to Replicate Artistic Stylescited by 6
  21. 2019Alternative Function Approximation Parameterizations for Solving Games: An Analysis of f-Regression Counterfactual Regret Minimizationcited by 5
  22. 2022Event Tables for Efficient Experience Replaycited by 4
  23. 2022Configurable Agent With Reward As Input: A Play-Style Continuum Generationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Policy Fusion for Adaptive and Customizable Reinforcement Learning Agents
  2. 2026Synthesizing Reactive Character Behaviors for Continuous Games via Programmatic Policy Search
  3. 2024Personalized Dynamic Difficulty Adjustment -- Imitation Learning Meets Reinforcement Learning
  4. 2026Android Coach: Improve Online Agentic Training Efficiency with Single State Multiple Actions
  5. 2024Autoverse: An Evolvable Game Language for Learning Robust Embodied Agents
  6. 2026Training a Conditioned Video Game Agent on a VLM Annotated Dataset
  7. 2026MimicAgent: Quadruped Skills via Prompt-to-Trajectory Generation
  8. 2025DiffFP: Learning Behaviors from Scratch via Diffusion-based Fictitious Play