mlsanityData to 7 Oct 2026

Multitasking Inhibits Semantic Drift

arXiv 2104.07219 ↗First posted 15 Apr 2021cs.CL cs.AI cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

When intelligent agents communicate to accomplish shared goals, how do these goals shape the agents' language? We study the dynamics of learning in latent language policies (LLPs), in which instructor agents generate natural-language subgoal descriptions and executor agents map these descriptions to low-level actions. LLPs can solve challenging long-horizon reinforcement learning problems and provide a rich model for studying task-oriented language use. But previous work has found that LLP training is prone to semantic drift (use of messages in ways inconsistent with their original natural language meanings). Here, we demonstrate theoretically and empirically that multitask training is an effective counter to this problem: we prove that multitask training eliminates semantic drift in a well-studied family of signaling games, and show that multitask training of neural LLPs in a complex strategy game reduces drift and while improving sample efficiency.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 30 of its 49 references that are in the map. First posted on arXiv 15 Apr 2021.

Builds on

30 of 49 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2020Learning to summarize from human feedbackcited by 2,562
  6. 2015Trust Region Policy Optimizationcited by 2,503
  7. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  8. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  9. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  10. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  11. 2016The Option-Critic Architecturecited by 433
  12. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
18 more
  1. 2016Deep Reinforcement Learning for Dialogue Generationcited by 286
  2. 2016Safe and Efficient Off-Policy Reinforcement Learningcited by 260
  3. 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 202
  4. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  5. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  6. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 97
  7. 2017Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialogcited by 86
  8. 2019BAM! Born-Again Multi-Task Networks for Natural Language Understandingcited by 85
  9. 2017Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learningcited by 77
  10. 2017Learning with Latent Languagecited by 75
  11. 2016Loss is its own Reward: Self-Supervision for Reinforcement Learningcited by 72
  12. 2020Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learningcited by 65
  13. 2013Sample Complexity of Multi-task Reinforcement Learningcited by 58
  14. 2020Countering Language Drift with Seeded Iterated Learningcited by 57
  15. 2019Countering Language Drift via Visual Groundingcited by 55
  16. 2019Hierarchical Decision Making by Generating and Following Natural Language Instructionscited by 30
  17. 2018Community Regularization of Visually-Grounded Dialogcited by 5
  18. 2020Communicative need modulates competition in language changecited by 3

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 3; 2023: 1; 2024: 1; 2025: 1.

8 papers.

  1. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 220
  2. 2021Skill Induction and Planning with Latent Languagecited by 99
  3. 2021Modeling Strong and Human-Like Gameplay with KL-Regularized Searchcited by 50
  4. 2022History Compression via Language Models in Reinforcement Learningcited by 39
  5. 2023Semantic HELM: A Human-Readable Memory for Reinforcement Learningcited by 17
  6. 2024Assessing the Zero-Shot Capabilities of LLMs for Action Evaluation in RLcited by 7
  7. 2022One-Shot Learning from a Demonstration with Hierarchical Latent Languagecited by 6
  8. 2025Why do AI agents communicate in human language?cited by 2