Transformers as Game Players: Provable In-context Game-playing Capabilities of Pre-trained Models
Not on the map. Why?
Abstract (arXiv)
The in-context learning (ICL) capability of pre-trained models based on the transformer architecture has received growing interest in recent years. While theoretical understanding has been obtained for ICL in reinforcement learning (RL), the previous results are largely confined to the single-agent setting. This work proposes to further explore the in-context learning capabilities of pre-trained transformer models in competitive multi-agent games, i.e., in-context game-playing (ICGP). Focusing on the classical two-player zero-sum games, theoretical guarantees are provided to demonstrate that pre-trained transformers can provably learn to approximate Nash equilibrium in an in-context manner for both decentralized and centralized learning settings. As a key part of the proof, constructional results are established to demonstrate that the transformer architecture is sufficiently rich to realize celebrated multi-agent game-playing algorithms, in particular, decentralized V-learning and centralized VI-ULCB.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 49 of its 69 references that are in the map. First posted on arXiv 13 Oct 2024.
Builds on
49 of 69 references.
References · 49
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2023GPT-4 Technical Reportcited by 18,132
- 2020End-to-End Object Detection with Transformerscited by 7,384
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
- 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
- 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
- 2022Transformers learn in-context by gradient descentcited by 714
- 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
37 more
- 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
- 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
- 2017Minimax Regret Bounds for Reinforcement Learningcited by 500
- 2019Are Transformers universal approximators of sequence-to-sequence functions?cited by 404
- 2023Trained Transformers Learn Linear Models In-Contextcited by 331
- 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 291
- 2023Transformers learn to implement preconditioned gradient descent for in-context learningcited by 274
- 2023Looped Transformers as Programmable Computerscited by 224
- 2023Transformers as Algorithms: Generalization and Stability in In-context Learningcited by 220
- 2015Fast Convergence of Regularized Learning in Gamescited by 149
- 2023Pretraining task diversity and the emergence of non-Bayesian in-context learning for regressioncited by 138
- 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 137
- 2017On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learningcited by 136
- 2023The mechanistic basis of data dependence and abrupt learning in an in-context classification taskcited by 115
- 2020Provable Self-Play Algorithms for Competitive Reinforcement Learningcited by 112
- 2023In-Context Convergence of Transformerscited by 112
- 2020Near-Optimal Reinforcement Learning with Self-Playcited by 108
- 2020A Sharp Analysis of Model-based Reinforcement Learning with Self-Playcited by 104
- 2020Learning Zero-Sum Simultaneous-Move Markov Games Using Function Approximation and Correlated Equilibriumcited by 100
- 2023Uncovering mesa-optimization algorithms in Transformerscited by 94
- 2023How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?cited by 93
- 2020Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexitycited by 91
- 2021Near-Optimal No-Regret Learning in General Gamescited by 85
- 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 82
- 2023How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representationscited by 72
- 2021When Can We Learn General-Sum Markov Games with a Large Number of Players Sample-Efficiently?cited by 63
- 2023Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Contextcited by 63
- 2021The Power of Exploiter: Provable Multi-Agent RL in Large State Spacescited by 48
- 2021Towards General Function Approximation in Zero-Sum Markov Gamescited by 42
- 2021Provably Efficient Reinforcement Learning in Decentralized General-Sum Markov Gamescited by 42
- 2022Pessimistic Minimax Value Iteration: Provably Efficient Equilibrium Learning from Offline Datasetscited by 42
- 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 36
- 2023Breaking the Curse of Multiagency: Provably Efficient Decentralized Multi-Agent RL with Function Approximationcited by 29
- 2023Breaking the Curse of Multiagents in a Large State Space: RL in Markov Games with Independent Linear Function Approximationcited by 26
- 2023Emergence of In-Context Reinforcement Learning from Noise Distillationcited by 26
- 2023In-Context Reinforcement Learning for Variable Action Spacescited by 23
- 2022A Self-Play Posterior Sampling Algorithm for Zero-Sum Markov Gamescited by 14
Built on by
Citing papers per year: 2026: 1.
1 paper.