mlsanityData to 14 Oct 2026

Towards Interactive Language Modeling

arXiv 2112.11911 ↗First posted 14 Dec 2021cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Interaction between caregivers and children plays a critical role in human language acquisition and development. Given this observation, it is remarkable that explicit interaction plays little to no role in artificial language modeling -- which also targets the acquisition of human language, yet by artificial models. Moreover, an interactive approach to language modeling has the potential to make language models substantially more versatile and to considerably impact downstream applications. Motivated by these considerations, we pioneer the space of interactive language modeling. As a first contribution we present a road map in which we detail the steps that need to be taken towards interactive language modeling. We then lead by example and take the first steps on this road map, showing the initial feasibility of our approach. As such, this work aims to be the start of a larger research agenda on interactive language modeling.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 19 of its 61 references that are in the map. First posted on arXiv 14 Dec 2021.

Builds on

19 of 61 references.

References · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  6. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  7. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
  8. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 335
  9. 2019Compositionality decomposed: how do neural networks generalise?cited by 265
  10. 2019Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacementcited by 211
  11. 2017Teacher-Student Curriculum Learningcited by 210
  12. 2018Ranking Sentences for Extractive Summarization with Reinforcement Learningcited by 128
7 more
  1. 2021Lifelong Pretraining: Continually Adapting Language Models to Emerging Corporacited by 115
  2. 2018BanditSum: Extractive Summarization as a Contextual Banditcited by 46
  3. 2020Co-evolution of language and agents in referential gamescited by 18
  4. 2021NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environmentcited by 15
  5. 2021Can Transformers Jump Around Right in Natural Language? Assessing Performance Transfer from SCANcited by 14
  6. 2020Internal and external pressures on language emergence: least effort, object constancy and frequencycited by 13
  7. 2020The Grammar of Emergent Languagescited by 9

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 1; 2025: 2.

4 papers.

  1. 2022Evaluating Human-Language Model Interactioncited by 92
  2. 2024Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrationscited by 12
  3. 2025Teacher Demonstrations in a BabyLM's Zone of Proximal Development for Contingent Multi-Turn Interactioncited by 9
  4. 2025Unsupervised Acquisition of Discrete Grammatical Categoriescited by 1