# What game are we playing? End-to-end learning in normal and extensive form games

arXiv: https://arxiv.org/abs/1805.02777 · HTML: https://mlsanity.com/paper/1805.02777/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2018.

## Built on by (most cited first, 22 of 22)

- [DC3: A learning method for optimization with hard constraints](https://mlsanity.com/paper/2104.12225/) (2021)
- [A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Games](https://mlsanity.com/paper/2206.05825/) (2022)
- [Modeling Strong and Human-Like Gameplay with KL-Regularized Search](https://mlsanity.com/paper/2112.07544/) (2021)
- [CombOptNet: Fit the Right NP-Hard Problem by Learning Integer Programming Constraints](https://mlsanity.com/paper/2105.02343/) (2021)
- [Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimization](https://mlsanity.com/paper/2105.08881/) (2021)
- [How Bad is Top-K Recommendation under Competing Content Creators?](https://mlsanity.com/paper/2302.01971/) (2023)
- [Model-Free Online Learning in Unknown Sequential Decision Making Problems and Games](https://mlsanity.com/paper/2103.04539/) (2021)
- [Turbocharging Solution Concepts: Solving NEs, CEs and CCEs with Neural Equilibrium Solvers](https://mlsanity.com/paper/2210.09257/) (2022)
- [Differentiation Through Black-Box Quadratic Programming Solvers](https://mlsanity.com/paper/2410.06324/) (2024)
- [Adversarially Robust Learning for Security-Constrained Optimal Power Flow](https://mlsanity.com/paper/2111.06961/) (2021)
- [Ensuring DNN Solution Feasibility for Optimization Problems with Convex Constraints and Its Application to DC Optimal Power Flow Problems](https://mlsanity.com/paper/2112.08091/) (2021)
- [Is Nash Equilibrium Approximator Learnable?](https://mlsanity.com/paper/2108.07472/) (2021)
- [GenCO: Generating Diverse Designs with Combinatorial Constraints](https://mlsanity.com/paper/2310.02442/) (2023)
- [Data Driven VRP: A Neural Network Model to Learn Hidden Preferences for VRP](https://mlsanity.com/paper/2108.04578/) (2021)
- [Fail-Safe Adversarial Generative Imitation Learning](https://mlsanity.com/paper/2203.01696/) (2022)
- [Are Equivariant Equilibrium Approximators Beneficial?](https://mlsanity.com/paper/2301.11481/) (2023)
- [Deep Incentive Design with Differentiable Equilibrium Blocks](https://mlsanity.com/paper/2603.07705/) (2026)
- [Active Inverse Learning in Stackelberg Trajectory Games](https://mlsanity.com/paper/2308.08017/) (2023)
- [Differentiation of Multi-objective Data-driven Decision Pipeline](https://mlsanity.com/paper/2406.00779/) (2024)
- [Operator Splitting for Learning to Predict Equilibria in Convex Games](https://mlsanity.com/paper/2106.00906/) (2021)
- [From Behavioral Theories to Econometrics: Inferring Preferences of Human Agents from Data on Repeated Interactions](https://mlsanity.com/paper/2112.15151/) (2021)
- [Semiparametric Efficient Bilevel Gradient Estimation](https://mlsanity.com/paper/2605.21341/) (2026)

## Abstract (arXiv)

Although recent work in AI has made great progress in solving large, zero-sum, extensive-form games, the underlying assumption in most past work is that the parameters of the game itself are known to the agents. This paper deals with the relatively under-explored but equally important "inverse" setting, where the parameters of the underlying game are not known to all agents, but must be learned through observations. We propose a differentiable, end-to-end learning framework for addressing this task. In particular, we consider a regularized version of the game, equivalent to a particular form of quantal response equilibrium, and develop 1) a primal-dual Newton method for finding such equilibrium points in both normal and extensive form games; and 2) a backpropagation method that lets us analytically compute gradients of all relevant game parameters through the solution itself. This ultimately lets us learn the game by training in an end-to-end fashion, effectively by integrating a "differentiable game solver" into the loop of larger deep network architectures. We demonstrate the effectiveness of the learning method in several settings including poker and security game tasks.