mlsanityData to 14 Oct 2026

Keep CALM and Explore: Language Models for Action Generation in Text-based Games

arXiv 2010.02903 ↗First posted 6 Oct 2020cs.CLcited by 108

Not on the map. Why?

Cited by 28 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Text-based games present a unique challenge for autonomous agents to operate in natural language and handle enormous action spaces. In this paper, we propose the Contextual Action Language Model (CALM) to generate a compact set of action candidates at each game state. Our key insight is to train language models on human gameplay, where people demonstrate linguistic priors and a general game sense for promising actions conditioned on game history. We combine CALM with a reinforcement learning agent which re-ranks the generated action candidates to maximize in-game rewards. We evaluate our approach using the Jericho benchmark, on games unseen by CALM during training. Our method obtains a 69% relative improvement in average game score over the previous state-of-the-art model. Surprisingly, on half of these games, CALM is competitive with or better than other models that have access to ground truth admissible actions. Code and data are available at https://github.com/princeton-nlp/calm-textgame.

Key facts. Cited by 108 papers in the map (data through 14 Oct 2026). First posted on arXiv 6 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 13; 2022: 21; 2023: 28; 2024: 21; 2025: 18; 2026: 7.

108 papers.

Sort:

By area:

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  3. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  4. 2023Cognitive Architectures for Language Agentscited by 429
  5. 2024Understanding the planning of LLM agents: A surveycited by 421
  6. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 327
  7. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 260
  8. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 238
  9. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 221
  10. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 187
  11. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 182
  12. 2023SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Taskscited by 168
38 more
  1. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 160
  2. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 116
  3. 2024A Survey on Large Language Model-Based Game Agentscited by 112
  4. 2024AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agentscited by 102
  5. 2023CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalizationcited by 95
  6. 2024Large Language Models and Games: A Survey and Roadmapcited by 93
  7. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 88
  8. 2021What Would Jiminy Cricket Do? Towards Agents That Behave Morallycited by 57
  9. 2022History Compression via Language Models in Reinforcement Learningcited by 39
  10. 2024Agent-Oriented Planning in Multi-Agent Systemscited by 36
  11. 2025VideoGameBench: Can Vision-Language Models complete popular video games?cited by 36
  12. 2022Dungeons and Dragons as a Dialog Challenge for Artificial Intelligencecited by 35
  13. 2024BAGEL: Bootstrapping Agents by Guiding Exploration with Languagecited by 34
  14. 2025Ignore the KL Penalty! Boosting Exploration on Critical Tokens to Enhance RL Fine-Tuningcited by 34
  15. 2022Aligning to Social Norms and Values in Interactive Narrativescited by 32
  16. 2023Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agentscited by 32
  17. 2021Learning Knowledge Graph-based World Models of Textual Environmentscited by 31
  18. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  19. 2022CAPE: Corrective Actions from Precondition Errors using Large Language Modelscited by 30
  20. 2023Augmenting Autotelic Agents with Large Language Modelscited by 30
  21. 2021Adaptive Information Seeking for Open-Domain Question Answeringcited by 29
  22. 2021Language Models are Few-Shot Butlerscited by 27
  23. 2022MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learnedcited by 26
  24. 2025Sparks of Science: Hypothesis Generation Using Structured Paper Datacited by 25
  25. 2022Multi-Stage Episodic Control for Strategic Exploration in Text Gamescited by 22
  26. 2022Few-shot Subgoal Planning with Language Modelscited by 22
  27. 2025Monte Carlo Planning with Large Language Model for Text-Based Game Agentscited by 22
  28. 2021Pre-trained Language Models as Prior Knowledge for Playing Text-based Gamescited by 21
  29. 2021Reading and Acting while Blindfolded: The Need for Semantics in Text Game Agentscited by 20
  30. 2024Large Action Models: From Inception to Implementationcited by 18
  31. 2021A Systematic Survey of Text Worlds as Embodied Natural Language Environmentscited by 17
  32. 2023Semantic HELM: A Human-Readable Memory for Reinforcement Learningcited by 17
  33. 2021Inherently Explainable Reinforcement Learning in Natural Languagecited by 16
  34. 2024Player-Driven Emergence in LLM-Driven Game Narrativecited by 16
  35. 2023An Appraisal-Based Chain-Of-Emotion Architecture for Affective Language Model Game Agentscited by 15
  36. 2024Recover: A Neuro-Symbolic Framework for Failure Detection and Recoverycited by 15
  37. 2021A Survey of Text Games for Reinforcement Learning informed by Natural Languagecited by 14
  38. 2022Recent Advances in Neural Text Generation: A Task-Agnostic Surveycited by 14

50 most cited of 108 All 108 (JSON)