mlsanityData to 14 Oct 2026

Behavior Cloned Transformers are Neurosymbolic Reasoners

arXiv 2210.07382 ↗First posted 13 Oct 2022cs.CL cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

In this work, we explore techniques for augmenting interactive agents with information from symbolic modules, much like humans use tools like calculators and GPS systems to assist with arithmetic and navigation. We test our agent's abilities in text games -- challenging benchmarks for evaluating the multi-step reasoning abilities of game agents in grounded, language-based environments. Our experimental study indicates that injecting the actions from these symbolic modules into the action space of a behavior cloned transformer agent increases performance on four text game benchmarks that test arithmetic, navigation, sorting, and common sense reasoning by an average of 22%, allowing an agent to reach the highest possible performance on unseen games. This action injection technique is easily extended to new agents, environments, and symbolic modules.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 34 of its 43 references that are in the map. First posted on arXiv 13 Oct 2022.

Builds on

34 of 43 references.

References · 34

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  3. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
  4. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  5. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  6. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  7. 2018Behavioral Cloning from Observationcited by 492
  8. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 327
  9. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 214
  10. 2021Societal Biases in Language Generation: Progress and Challengescited by 202
  11. 2019Interactive Fiction Games: A Colossal Adventurecited by 168
  12. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
22 more
  1. 2020Keep CALM and Explore: Language Models for Action Generation in Text-based Gamescited by 108
  2. 2019Evaluating Commonsense in Pre-trained Language Modelscited by 95
  3. 2020Graph Constrained Reinforcement Learning for Natural Language Action Spacescited by 80
  4. 2018Learn What Not to Learn: Action Elimination with Deep Reinforcement Learningcited by 77
  5. 2020Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graphcited by 57
  6. 2020Learning Dynamic Belief Graphs to Generalize on Text-Based Gamescited by 49
  7. 2021PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D Worldcited by 47
  8. 2019NLProlog: Reasoning with Weak Unification for Question Answering in Natural Languagecited by 37
  9. 2020Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselinescited by 37
  10. 2018Counting to Explore and Generalize in Text-based Gamescited by 36
  11. 2020How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worldscited by 35
  12. 2021Learning Knowledge Graph-based World Models of Textual Environmentscited by 31
  13. 2021Neuro-Symbolic Reinforcement Learning with First-Order Logiccited by 30
  14. 2020Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Gamescited by 22
  15. 2021Reading and Acting while Blindfolded: The Need for Semantics in Text Game Agentscited by 20
  16. 2021A Systematic Survey of Text Worlds as Embodied Natural Language Environmentscited by 17
  17. 2021A Survey of Text Games for Reinforcement Learning informed by Natural Languagecited by 14
  18. 2022TextWorldExpress: Simulating Text Games at One Million Steps Per Secondcited by 13
  19. 2022Asking for Knowledge: Training RL Agents to Query External Knowledge Using Languagecited by 10
  20. 2021Case-based Reasoning for Better Generalization in Textual Reinforcement Learningcited by 8
  21. 2021LOA: Logical Optimal Actions for Text-based Interaction Gamescited by 6
  22. 2021Eye of the Beholder: Improved Relation Generalization for Text-based Reinforcement Learning Agentscited by 4

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 5; 2025: 1.

10 papers.

  1. 2023Augmented Language Models: a Surveycited by 403
  2. 2023Tool Learning with Foundation Modelscited by 302
  3. 2025Neuro-Symbolic AI in 2024: A Systematic Reviewcited by 38
  4. 2024Language Guided Exploration for RL Agents in Text Environmentscited by 9
  5. 2024STARLING: Self-supervised Training of Text-based Reinforcement Learning Agent with Large Language Modelscited by 8
  6. 2022MURMUR: Modular Multi-Step Reasoning for Semi-Structured Data-to-Text Generationcited by 7
  7. 2024EXPLORER: Exploration-guided Reasoning for Textual Reinforcement Learningcited by 3
  8. 2024A Mixture-of-Experts Approach to Few-Shot Task Transfer in Open-Ended Text Worldscited by 2
  9. 2023Self-Supervised Behavior Cloned Transformers are Path Crawlers for Text Gamescited by 1
  10. 2024On the Effects of Fine-tuning Language Models for Text-Based Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Large Language Models Are Neurosymbolic Reasoners
  2. 2025The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mind
  3. 2026Lifting Traces to Logic: Programmatic Skill Induction with Neuro-Symbolic Learning for Long-Horizon Agentic Tasks
  4. 2023Language Decision Transformers with Exponential Tilt for Interactive Text Environments
  5. 2026GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents
  6. 2020Benchmarking End-to-End Behavioural Cloning on Video Games
  7. 2026Up and Down the Abstraction Ladder: Code-Based Skills for Language Agents
  8. 2025IPR-1: Interactive Physical Reasoner