# Approximating Discontinuous Nash Equilibrial Values of Two-Player General-Sum Differential Games

arXiv: https://arxiv.org/abs/2207.01773 · HTML: https://mlsanity.com/paper/2207.01773/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 46 references that are in the map.
- First posted on arXiv 5 Jul 2022.

## Builds on (most cited first, 21 of 21)

- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [Learning with Opponent-Learning Awareness](https://mlsanity.com/paper/1709.04326/) (2017)
- [Optimal approximation of piecewise smooth functions using deep ReLU neural networks](https://mlsanity.com/paper/1709.05289/) (2017)
- [Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning](https://mlsanity.com/paper/2206.15378/) (2022)
- [Combining Deep Reinforcement Learning and Search for Imperfect-Information Games](https://mlsanity.com/paper/2007.13544/) (2020)
- [Deep Counterfactual Regret Minimization](https://mlsanity.com/paper/1811.00164/) (2018)
- [DeepReach: A Deep Learning Approach to High-Dimensional Reachability](https://mlsanity.com/paper/2011.02082/) (2020)
- [Convergence of the Deep BSDE Method for Coupled FBSDEs](https://mlsanity.com/paper/1811.01165/) (2018)
- [Adaptive Deep Learning for High-Dimensional Hamilton-Jacobi-Bellman Equations](https://mlsanity.com/paper/1907.05317/) (2019)
- [Efficient Iterative Linear-Quadratic Approximations for Nonlinear Multi-Player General-Sum Differential Games](https://mlsanity.com/paper/1909.04694/) (2019)
- [When Humans Aren't Optimal: Robots that Collaborate with Risk-Aware Humans](https://mlsanity.com/paper/2001.04377/) (2020)
- [Probabilistic Prediction of Interactive Driving Behavior via Hierarchical Inverse Reinforcement Learning](https://mlsanity.com/paper/1809.02926/) (2018)
- [A neural network based policy iteration algorithm with global H2-superlinear convergence for stochastic games on domains](https://mlsanity.com/paper/1906.02304/) (2019)
- [Bayesian Persuasive Driving](https://mlsanity.com/paper/1809.09735/) (2018)
- [When Shall I Be Empathetic? The Utility of Empathetic Parameter Estimation in Multi-Agent Interactions](https://mlsanity.com/paper/2011.02047/) (2020)
- [Seeking Nash equilibrium in non-cooperative differential games](https://mlsanity.com/paper/2003.01098/) (2020)
- [A systematic process for evaluating structured perfect Bayesian equilibria in dynamic games with asymmetric information](https://mlsanity.com/paper/1508.06269/) (2015)
- [Structured Perfect Bayesian Equilibrium in Infinite Horizon Dynamic Games with Asymmetric Information](https://mlsanity.com/paper/1609.04221/) (2016)

## Built on by (most cited first, 5 of 5)

- [Value Approximation for Two-Player General-Sum Differential Games with State Constraints](https://mlsanity.com/paper/2311.16520/) (2023)
- [Pontryagin Neural Operator for Solving Parametric General-Sum Differential Games](https://mlsanity.com/paper/2401.01502/) (2024)
- [Learning Nash Equilibrial Hamiltonian for Two-Player Collision-Avoiding Interactions](https://mlsanity.com/paper/2503.07013/) (2025)
- [PACE: A Framework for Learning and Control in Linear Incomplete-Information Differential Games](https://mlsanity.com/paper/2504.17128/) (2025)
- [Peer-Aware Cost Estimation in Nonlinear General-Sum Dynamic Games for Mutual Learning and Intent Inference](https://mlsanity.com/paper/2504.17129/) (2025)

## Abstract (arXiv)

Finding Nash equilibrial policies for two-player differential games requires solving Hamilton-Jacobi-Isaacs (HJI) PDEs. Self-supervised learning has been used to approximate solutions of such PDEs while circumventing the curse of dimensionality. However, this method fails to learn discontinuous PDE solutions due to its sampling nature, leading to poor safety performance of the resulting controllers in robotics applications when player rewards are discontinuous. This paper investigates two potential solutions to this problem: a hybrid method that leverages both supervised Nash equilibria and the HJI PDE, and a value-hardening method where a sequence of HJIs are solved with a gradually hardening reward. We compare these solutions using the resulting generalization and safety performance in two vehicle interaction simulation studies with 5D and 9D state spaces, respectively. Results show that with informative supervision (e.g., collision and near-collision demonstrations) and the low cost of self-supervised learning, the hybrid method achieves better safety performance than the supervised, self-supervised, and value hardening approaches on equal computational budget. Value hardening fails to generalize in the higher-dimensional case without informative supervision. Lastly, we show that the neural activation function needs to be continuously differentiable for learning PDEs and its choice can be case dependent.