# Play Like Champions: Counterfactual Feedback Generation in Latent Space

arXiv: https://arxiv.org/abs/2607.00190 · HTML: https://mlsanity.com/paper/2607.00190/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 48 references that are in the map.
- First posted on arXiv 30 Jun 2026.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [StarCraft II: A New Challenge for Reinforcement Learning](https://mlsanity.com/paper/1708.04782/) (2017)
- [SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2212.07489/) (2022)
- [Aligning Superhuman AI with Human Behavior: Chess as a Model System](https://mlsanity.com/paper/2006.01855/) (2020)
- [Guided Variational Autoencoder for Disentanglement Learning](https://mlsanity.com/paper/2004.01255/) (2020)
- [Learning coordinated badminton skills for legged manipulators](https://mlsanity.com/paper/2505.22974/) (2025)
- [Counterfactual Explanations as Interventions in Latent Space](https://mlsanity.com/paper/2106.07754/) (2021)
- [Latent Traversals in Generative Models as Potential Flows](https://mlsanity.com/paper/2304.12944/) (2023)
- [Optimal transport maps for distribution preserving operations on latent spaces of Generative Models](https://mlsanity.com/paper/1711.01970/) (2017)
- [Using Reinforcement Learning for Load Testing of Video Games](https://mlsanity.com/paper/2201.06865/) (2022)
- [AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning](https://mlsanity.com/paper/2308.03526/) (2023)
- [Counterfactual Explanations via Riemannian Latent Space Traversal](https://mlsanity.com/paper/2411.02259/) (2024)
- [A Sim-to-Real Deep Learning-based Framework for Autonomous Nano-drone Racing](https://mlsanity.com/paper/2312.08991/) (2023)
- [SC2EGSet: StarCraft II Esport Replay and Game-state Dataset](https://mlsanity.com/paper/2207.03428/) (2022)
- [Humanoid Whole-Body Badminton via an Annealed Reinforcement Learning Curriculum](https://mlsanity.com/paper/2511.11218/) (2025)
- [ggViz: Accelerating Large-Scale Esports Game Analysis](https://mlsanity.com/paper/2107.06495/) (2021)
- [Human-Like Goalkeeping in a Realistic Football Simulation: a Sample-Efficient Reinforcement Learning Approach](https://mlsanity.com/paper/2510.23216/) (2025)
- [F.A.C.U.L.: Language-Based Interaction with AI Companions in Gaming](https://mlsanity.com/paper/2511.13112/) (2025)
- [Carefully Structured Compression: Efficiently Managing StarCraft II Data](https://mlsanity.com/paper/2410.08659/) (2024)

## Abstract (arXiv)

Recent advances in reinforcement learning have produced superhuman agents across a wide range of competitive games. As a byproduct, researchers have begun studying how these agents play, extracting behavioral representations, analyzing decision structure, and modeling the latent geometry of expert performance. However, this growing body of work has overwhelmingly focused on defeating human players rather than providing feedback, leaving a critical gap in creating model solutions to improve human players. Unlike chess and Go, where AI has become integral to player training, real-time strategy (RTS) games lack principled frameworks for translating expert knowledge into actionable feedback. We introduce Latent Maps of Performance, a framework for counterfactual path generation. We focus on StarCraft~II data to model player improvement as an algorithmic recourse within a learned representation space. As inspiration for our work, we have looked at the championship model used in sports science. We trained a Guided Variational Autoencoder model on 23,305 professional tournament replays, enabling counterfactual traversal between losing and winning gameplay profiles. To fulfill our goal, we have devised and verified four traversal strategies on out-of-distribution (OOD) data randomly sampled from a dataset of amateur replays, namely linear interpolation, iterative optimal transport, density-regularized gradient ascent, and neural flow matching, each designed to generate multi-step improvement trajectories that remain grounded in observed expert behavior while moving a player's profile toward winning configurations. Feedback is extracted at multiple granularities to support players at different stages of improvement. Finally, we conclude that there is a trade-off between the path-finding methods we employ and hope that future research will focus on developing model solutions for human improvement.