# Towards Sustainable Investment Policies Informed by Opponent Shaping

arXiv: https://arxiv.org/abs/2602.11829 · HTML: https://mlsanity.com/paper/2602.11829/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 12 Feb 2026.

## Builds on (most cited first, 11 of 11)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Learning with Opponent-Learning Awareness](https://mlsanity.com/paper/1709.04326/) (2017)
- [Stable Opponent Shaping in Differentiable Games](https://mlsanity.com/paper/1811.08469/) (2018)
- [The AI Economist: Optimal Economic Policy Design via Two-level Deep Reinforcement Learning](https://mlsanity.com/paper/2108.02755/) (2021)
- [LOQA: Learning with Opponent Q-Learning Awareness](https://mlsanity.com/paper/2405.01035/) (2024)
- [Translucent Players: Explaining Cooperative Behavior in Social Dilemmas](https://mlsanity.com/paper/1410.3363/) (2014)
- [InvestESG: A multi-agent reinforcement learning benchmark for studying climate investment as a social dilemma](https://mlsanity.com/paper/2411.09856/) (2024)
- [Carbon Market Simulation with Adaptive Mechanism Design](https://mlsanity.com/paper/2406.07875/) (2024)

## Abstract (arXiv)

Addressing climate change requires global coordination, yet rational economic actors often prioritize immediate gains over collective welfare, resulting in social dilemmas. InvestESG is a recently proposed multi-agent simulation that captures the dynamic interplay between investors and companies under climate risk. We provide a formal characterization of the conditions under which InvestESG exhibits an intertemporal social dilemma, deriving theoretical thresholds at which individual incentives diverge from collective welfare. Building on this, we apply Advantage Alignment, a scalable opponent shaping algorithm shown to be effective in general-sum games, to influence agent learning in InvestESG. We offer theoretical insights into why Advantage Alignment systematically favors socially beneficial equilibria by biasing learning dynamics toward cooperative outcomes. Our results demonstrate that strategically shaping the learning processes of economic agents can result in better outcomes that could inform policy mechanisms to better align market incentives with long-term sustainability goals.