mlsanityData to 7 Oct 2026

A Song of Ice and Fire: Analyzing Textual Autotelic Agents in ScienceWorld

arXiv 2302.05244 ↗First posted 10 Feb 2023cs.AI cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Building open-ended agents that can autonomously discover a diversity of behaviours is one of the long-standing goals of artificial intelligence. This challenge can be studied in the framework of autotelic RL agents, i.e. agents that learn by selecting and pursuing their own goals, self-organizing a learning curriculum. Recent work identified language as a key dimension of autotelic learning, in particular because it enables abstract goal sampling and guidance from social peers for hindsight relabelling. Within this perspective, we study the following open scientific questions: What is the impact of hindsight feedback from a social peer (e.g. selective vs. exhaustive)? How can the agent learn from very rare language goal examples in its experience replay? How can multiple forms of exploration be combined, and take advantage of easier goals as stepping stones to reach harder ones? To address these questions, we use ScienceWorld, a textual environment with rich abstract and combinatorial physics. We show the importance of selectivity from the social peer's feedback; that experience replay needs to over-sample examples of rare goals; and that following self-generated goal sequences where the agent's competence is intermediate leads to significant improvements in final performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 44 references that are in the map. First posted on arXiv 10 Feb 2023.

Builds on

32 of 44 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  3. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  4. 2017Hindsight Experience Replaycited by 1,096
  5. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  6. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  7. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  8. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  9. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  10. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  11. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 298
  12. 2020First return, then explorecited by 256
20 more
  1. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  2. 2019Interactive Fiction Games: A Colossal Adventurecited by 165
  3. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  4. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 141
  5. 2020Keep CALM and Explore: Language Models for Action Generation in Text-based Gamescited by 108
  6. 2015Deep Reinforcement Learning with a Natural Language Action Spacecited by 88
  7. 2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 86
  8. 2020Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Surveycited by 83
  9. 2017Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learningcited by 79
  10. 2018CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learningcited by 78
  11. 2020Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Explorationcited by 68
  12. 2013Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robotscited by 56
  13. 2022Semantic Exploration from Language Abstractions and Pretrained Representationscited by 51
  14. 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
  15. 2021ELLA: Exploration through Learned Language Abstractioncited by 42
  16. 2018Unsupervised Learning of Goal Spaces for Intrinsically Motivated Goal Explorationcited by 23
  17. 2020Exploration Based Language Learning for Text-Based Gamescited by 18
  18. 2021A Systematic Survey of Text Worlds as Embodied Natural Language Environmentscited by 17
  19. 2022Language and Culture Internalisation for Human-Like Autotelic AIcited by 14
  20. 2020How To Avoid Being Eaten By a Grue: Exploration Strategies for Text-Adventure Agentscited by 10

Built on by

Not cited yet.