mlsanityData to 7 Oct 2026

Countering Language Drift with Seeded Iterated Learning

arXiv 2003.12694 ↗First posted 28 Mar 2020cs.AI cs.CLcited by 57

Not on the map. Why?

Abstract (arXiv)

Pretraining on human corpus and then finetuning in a simulator has become a standard pipeline for training a goal-oriented dialogue agent. Nevertheless, as soon as the agents are finetuned to maximize task completion, they suffer from the so-called language drift phenomenon: they slowly lose syntactic and semantic properties of language as they only focus on solving the task. In this paper, we propose a generic approach to counter language drift called Seeded iterated learning (SIL). We periodically refine a pretrained student agent by imitating data sampled from a newly generated teacher agent. At each time step, the teacher is created by copying the student agent, before being finetuned to maximize task completion. SIL does not require external syntactic constraint nor semantic knowledge, making it a valuable task-agnostic finetuning protocol. We evaluate SIL in a toy-setting Lewis Game, and then scale it up to the translation game with natural language. In both settings, SIL helps counter language drift as well as it improves the task completion compared to baselines.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 6; 2023: 17; 2024: 20; 2025: 4; 2026: 2.

57 papers.

Sort:

By area:

  1. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  2. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  3. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
  4. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  5. 2022SINE: SINgle Image Editing with Text-to-Image Diffusion Modelscited by 159
  6. 2023Cones: Concept Neurons in Diffusion Models for Customized Generationcited by 150
  7. 2024WARM: On the Benefits of Weight Averaged Reward Modelscited by 145
  8. 2023A Recipe for Watermarking Diffusion Modelscited by 136
  9. 2023ITI-GEN: Inclusive Text-to-Image Generationcited by 64
  10. 2024DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalizationcited by 60
  11. 2023Customizing 360-Degree Panoramas through Text-to-Image Diffusion Modelscited by 47
  12. 2024WARP: On the Benefits of Weight Averaged Rewarded Policiescited by 42
38 more
  1. 2022Fortuitous Forgetting in Connectionist Networkscited by 39
  2. 2023Language Model Alignment with Elastic Resetcited by 34
  3. 2023All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Modelscited by 33
  4. 2024Attention Calibration for Disentangled Text-to-Image Personalizationcited by 31
  5. 2023AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatarscited by 29
  6. 2024SleeperMark: Towards Robust Watermark against Fine-Tuning Text-to-image Diffusion Modelscited by 26
  7. 2023ViCo: Plug-and-play Visual Condition for Personalized Text-to-image Generationcited by 23
  8. 2021Iterated learning for emergent systematicity in VQAcited by 21
  9. 2024Countering Reward Over-optimization in LLM with Demonstration-Guided Reinforcement Learningcited by 20
  10. 2021Multi-label Iterated Learning for Image Classification with Label Ambiguitycited by 19
  11. 2024Eliciting the Priors of Large Language Models using Iterated In-Context Learningcited by 19
  12. 2023NewMove: Customizing text-to-video models with novel motionscited by 18
  13. 2024Bias Amplification in Language Model Evolution: An Iterated Learning Perspectivecited by 17
  14. 2024A Review of the Applications of Deep Learning-Based Emergent Communicationcited by 17
  15. 2023Improving Compositional Generalization Using Iterated Learning and Simplicial Embeddingscited by 15
  16. 2023Generate Anything Anywhere in Any Scenecited by 14
  17. 2021The Emergence of the Shape Bias Results from Communicative Efficiencycited by 13
  18. 2025Fine-Tuning Visual Autoregressive Models for Subject-Driven Generationcited by 13
  19. 2023Communication Drives the Emergence of Language Universals in Neural Agents: Evidence from the Word-order/Case-marking Trade-offcited by 12
  20. 2023Towards More Human-like AI Communication: A Review of Emergent Communication Researchcited by 12
  21. 2024ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflectionscited by 12
  22. 2021Dynamic population-based meta-learning for multi-agent communication with natural languagecited by 10
  23. 2024Evolving Storytelling: Benchmarks and Methods for New Character Customization with Diffusion Modelscited by 9
  24. 2021Multitasking Inhibits Semantic Driftcited by 8
  25. 2023My3DGen: A Scalable Personalized 3D Generative Modelcited by 8
  26. 2021On Slowly-varying Non-stationary Banditscited by 7
  27. 2023The Big Data Myth: Using Diffusion Models for Dataset Generation to Train Deep Detection Modelscited by 7
  28. 2024U-VAP: User-specified Visual Appearance Personalization via Decoupled Self Augmentationcited by 7
  29. 2024lpNTK: Better Generalisation with Less Data via Sample Interaction During Learningcited by 6
  30. 2025Personalized Text-to-Image Generation with Auto-Regressive Modelscited by 6
  31. 2024Language Evolution with Deep Learningcited by 5
  32. 2024MyTimeMachine: Personalized Facial Age Transformationcited by 5
  33. 2022Learning to translate by learning to communicatecited by 4
  34. 2024DreamBlend: Advancing Personalized Fine-tuning of Text-to-Image Diffusion Modelscited by 4
  35. 2025Towards Developmentally Plausible Rewards: Communicative Success as a Learning Signal for Interactive Language Modelscited by 4
  36. 2021Streaming Self-Training via Domain-Agnostic Unlabeled Imagescited by 3
  37. 2024An iterated learning model of language change that mixes supervised and unsupervised learningcited by 3
  38. 2024PersonaMagic: Stage-Regulated High-Fidelity Face Customization with Tandem Equilibriumcited by 3

50 most cited of 57 All 57 (JSON)