# A Mixture-of-Experts Approach to Few-Shot Task Transfer in Open-Ended Text Worlds

arXiv: https://arxiv.org/abs/2405.06059 · HTML: https://mlsanity.com/paper/2405.06059/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in MoE in reinforcement learning and robotics, one of 12 directions in Mixture-of-experts.
- Builds on 30 of its 38 references that are in the map.
- First posted on arXiv 9 May 2024.

## Placed in

- Mixture-of-experts / [MoE in reinforcement learning and robotics](https://mlsanity.com/moe/moe-in-reinforcement-learning-and-robotics/)

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [Distral: Robust Multitask Reinforcement Learning](https://mlsanity.com/paper/1707.04175/) (2017)
- [Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark](https://mlsanity.com/paper/2304.03279/) (2023)
- [Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning](https://mlsanity.com/paper/1511.06342/) (2015)
- [Multi-Task Reinforcement Learning with Context-based Representations](https://mlsanity.com/paper/2102.06177/) (2021)
- [Equivalence Between Policy Gradients and Soft Q-Learning](https://mlsanity.com/paper/1704.06440/) (2017)
- [Interactive Fiction Games: A Colossal Adventure](https://mlsanity.com/paper/1909.05398/) (2019)
- [An Actor-Critic Algorithm for Sequence Prediction](https://mlsanity.com/paper/1607.07086/) (2016)
- [Learning to Speak and Act in a Fantasy Text Adventure Game](https://mlsanity.com/paper/1903.03094/) (2019)
- [Graph Constrained Reinforcement Learning for Natural Language Action Spaces](https://mlsanity.com/paper/2001.08837/) (2020)
- [Language Understanding for Text-based Games Using Deep Reinforcement Learning](https://mlsanity.com/paper/1506.08941/) (2015)
- [Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning](https://mlsanity.com/paper/1812.01628/) (2018)
- [Learning Dynamic Belief Graphs to Generalize on Text-Based Games](https://mlsanity.com/paper/2002.09127/) (2020)
- [How to Avoid Being Eaten by a Grue: Structured Exploration Strategies for Textual Worlds](https://mlsanity.com/paper/2006.07409/) (2020)
- [How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds](https://mlsanity.com/paper/2010.00685/) (2020)
- [AC-Teach: A Bayesian Actor-Critic Method for Policy Learning with an Ensemble of Suboptimal Teachers](https://mlsanity.com/paper/1909.04121/) (2019)
- [Enhancing Text-based Reinforcement Learning Agents with Commonsense Knowledge](https://mlsanity.com/paper/2005.00811/) (2020)
- [Behavior Cloned Transformers are Neurosymbolic Reasoners](https://mlsanity.com/paper/2210.07382/) (2022)
- [Knowledge-enhanced Agents for Interactive Text Games](https://mlsanity.com/paper/2305.05091/) (2023)
- [Story Shaping: Teaching Agents Human-like Behavior with Stories](https://mlsanity.com/paper/2301.10107/) (2023)
- [Discovering Generalizable Skills via Automated Generation of Diverse Tasks](https://mlsanity.com/paper/2106.13935/) (2021)
- [A Minimal Approach for Natural Language Action Space in Text-based Games](https://mlsanity.com/paper/2305.04082/) (2023)

## Built on by (most cited first, 2 of 2)

- [TALES: Text Adventure Learning Environment Suite](https://mlsanity.com/paper/2504.14128/) (2025)
- [Position: AI Is Not Ready for Strategic Conflicts](https://mlsanity.com/paper/2609.16189/) (2026)

## Abstract (arXiv)

Open-ended worlds are those in which there are no pre-specified goals or environmental reward signal. As a consequence, an agent must know how to perform a multitude of tasks. However, when a new task is presented to an agent, we expect it to be able to reuse some of what it knows from previous tasks to rapidly learn that new task. We introduce a novel technique whereby policies for different a priori known tasks are combined into a Mixture-of-Experts model with an attention mechanism across a mix of frozen and unfrozen experts. The model learns when to attend to frozen task-specific experts when appropriate and learns new experts to handle novel situations. We work in an open-ended text-based environment in which the agent is tasked with behaving like different types of character roles and must rapidly learn behaviors associated with new character role types. We show that our agent both obtains more rewards in the zero-shot setting, and discovers these rewards with greater sample efficiency in the few-shot learning settings.