mlsanityData to 14 Oct 2026

ChessGPT: Bridging Policy Learning and Language Modeling

arXiv 2306.09200 ↗First posted 15 Jun 2023cs.LG cs.AIcited by 75

Not on the map. Why?

Abstract (arXiv)

When solving decision-making tasks, humans typically depend on information from two key sources: (1) Historical policy data, which provides interaction replay from the environment, and (2) Analytical insights in natural language form, exposing the invaluable thought process or strategic considerations. Despite this, the majority of preceding research focuses on only one source: they either use historical replay exclusively to directly learn policy or value functions, or engaged in language model training utilizing mere language corpus. In this paper, we argue that a powerful autonomous agent should cover both sources. Thus, we propose ChessGPT, a GPT model bridging policy learning and language modeling by integrating data from these two sources in Chess games. Specifically, we build a large-scale game and language dataset related to chess. Leveraging the dataset, we showcase two model examples ChessCLIP and ChessGPT, integrating policy learning and language modeling. Finally, we propose a full evaluation framework for evaluating language model's chess ability. Experimental results validate our model and dataset's effectiveness. We open source our code, model, and dataset at https://github.com/waterhorse1/ChessGPT.

Key facts. Cited by 75 papers in the map (data through 14 Oct 2026). Builds on 38 of its 61 references that are in the map. First posted on arXiv 15 Jun 2023.

Builds on

38 of 61 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  8. 2017Deep reinforcement learning from human preferencescited by 3,912
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  10. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  12. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
26 more
  1. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  3. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  5. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  7. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  8. 2021Grounded Language-Image Pre-trainingcited by 1,156
  9. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  10. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  11. 2022A Generalist Agentcited by 808
  12. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  13. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  14. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  15. 2023Instruction Tuning with GPT-4cited by 686
  16. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  17. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  18. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 398
  19. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  20. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 255
  21. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 187
  22. 2023The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Datasetcited by 148
  23. 2021Acquisition of Chess Knowledge in AlphaZerocited by 123
  24. 2020Aligning Superhuman AI with Human Behavior: Chess as a Model Systemcited by 79
  25. 2021Chess as a Testbed for Language Model State Trackingcited by 72
  26. 2020The Chess Transformer: Mastering Play using Generative Language Modelscited by 37

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 27; 2025: 23; 2026: 21.

75 papers.

Sort:

By area:

  1. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 295
  2. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 205
  3. 2024A Survey on Large Language Model-Based Game Agentscited by 112
  4. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 99
  5. 2024ChatMusician: Understanding and Generating Music Intrinsically with LLMcited by 68
  6. 2025VideoWorld: Exploring Knowledge Learning from Unlabeled Videoscited by 43
  7. 2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 42
  8. 2024Benchmarks as Microscopes: A Call for Model Metrologycited by 35
  9. 2024Evidence of Learned Look-Ahead in a Chess-Playing Neural Networkcited by 34
  10. 2024BattleAgentBench: A Benchmark for Evaluating Cooperation and Competition Capabilities of Language Models in Multi-Agent Systemscited by 26
  11. 2024Maia-2: A Unified Model for Human-AI Alignment in Chesscited by 25
  12. 2024Learning to Discuss Strategically: A Case Study on One Night Ultimate Werewolfcited by 24
38 more
  1. 2024Easy2Hard-Bench: Standardized Difficulty Labels for Profiling LLM Performance and Generalizationcited by 24
  2. 2024A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarioscited by 23
  3. 2024PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videoscited by 22
  4. 2023Chain of Images for Intuitively Reasoningcited by 21
  5. 2024Transcendence: Generative Models Can Outperform The Experts That Train Themcited by 20
  6. 2024GLEE: A Unified Framework and Benchmark for Language-based Economic Environmentscited by 20
  7. 2024Natural Language Reinforcement Learningcited by 17
  8. 2024Reinforcing Language Agents via Policy Optimization with Action Decompositioncited by 16
  9. 2025Complete Chess Games Enable LLM Become A Chess Mastercited by 16
  10. 2025Game Theory Meets Large Language Models: A Systematic Survey with Taxonomy and New Frontierscited by 16
  11. 2023Assessing Logical Puzzle Solving in Large Language Models: Insights from a Minesweeper Case Studycited by 15
  12. 2025LLM CHESS: Benchmarking Reasoning and Instruction-Following in LLMs through Chesscited by 15
  13. 2025VisualSphinx: Large-Scale Synthetic Vision Logic Puzzles for RLcited by 12
  14. 2024Human-aligned Chess with a Bit of Searchcited by 11
  15. 2025Implicit Search via Discrete Diffusion: A Study on Chesscited by 11
  16. 2025Synthelite: Chemist-aligned and feasibility-aware synthesis planning with LLMscited by 10
  17. 2025ChessQA: Evaluating Large Language Models for Chess Understandingcited by 8
  18. 2024Bridging the Gap between Expert and Language Models: Concept-guided Chess Commentary Generation and Evaluationcited by 6
  19. 2025Exploring Large Language Models for Word Games:Who is the Spy?cited by 6
  20. 2024LLMs May Not Be Human-Level Players, But They Can Be Testers: Measuring Game Difficulty with LLM Agentscited by 5
  21. 2024Natural Language Reinforcement Learningcited by 5
  22. 2024SMAC-Hard: Enabling Mixed Opponent Strategy Script and Self-play on SMACcited by 5
  23. 2025PuzzlePlex: Benchmarking Foundation Models on Reasoning and Planning with Puzzlescited by 5
  24. 2026Distilling Game Code World Model Generation into Lightweight Large Language Modelscited by 5
  25. 2026Understanding Reasoning from Pretraining to Post-Trainingcited by 5
  26. 2024Read to Play (R2-Play): Decision Transformer with Multimodal Game Instructioncited by 4
  27. 2025Curious Causality-Seeking Agents Learn Meta Causal Worldcited by 4
  28. 2026OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamicscited by 4
  29. 2024Reasoning, Memorization, and Fine-Tuning Language Models for Non-Cooperative Gamescited by 3
  30. 2025From Entity-Centric to Goal-Oriented Graphs: Enhancing LLM Knowledge Retrieval in Minecraftcited by 3
  31. 2025ChessArena: A Chess Testbed for Evaluating Strategic Reasoning Capabilities of Large Language Modelscited by 3
  32. 2024RL-LLM-DT: An Automatic Decision Tree Generation Method Based on RL Evaluation and LLM Enhancementcited by 2
  33. 2025Agents Require Metacognitive and Strategic Reasoning to Succeed in the Coming Labor Marketscited by 2
  34. 2025Baba is LLM: Reasoning in a Game with Dynamic Rulescited by 2
  35. 2025HLSMAC: A New StarCraft Multi-Agent Challenge for High-Level Strategic Decision-Makingcited by 2
  36. 2025Shall We Play a Game? Language Models for Open-ended Wargamescited by 2
  37. 2024Aspect-based Sentiment Evaluation of Chess Moves (ASSESS): an NLP-based Method for Evaluating Chess Strategies from Textbookscited by 1
  38. 2024Exploring the Learning Capabilities of Language Models using LEVERWORLDScited by 1

50 most cited of 75 All 75 (JSON)