mlsanityData to 14 Oct 2026

Blank Language Models

arXiv 2002.03079 ↗First posted 8 Feb 2020cs.CL cs.LGcited by 43

Not on the map. Why?

Abstract (arXiv)

We propose Blank Language Model (BLM), a model that generates sequences by dynamically creating and filling in blanks. The blanks control which part of the sequence to expand, making BLM ideal for a variety of text editing and rewriting tasks. The model can start from a single blank or partially completed text with blanks at specified locations. It iteratively determines which word to place in a blank and whether to insert new blanks, and stops generating when no blanks are left to fill. BLM can be efficiently trained using a lower bound of the marginal data likelihood. On the task of filling missing text snippets, BLM significantly outperforms all other baselines in terms of both accuracy and fluency. Experiments on style transfer and damaged ancient text restoration demonstrate the potential of this framework for a wide range of applications.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 11; 2022: 12; 2023: 8; 2024: 5; 2025: 3; 2026: 4.

43 papers.

Sort:

By area:

  1. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  2. 2023Augmented Language Models: a Surveycited by 403
  3. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  4. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  5. 2022PEER: A Collaborative Language Modelcited by 86
  6. 2022DeepStruct: Pretraining of Language Models for Structure Predictioncited by 45
  7. 2022Phrase-level Textual Adversarial Attack with Label Preservationcited by 23
  8. 2021Conformal prediction for text infilling and part-of-speech predictioncited by 22
  9. 2026DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvascited by 19
  10. 2021Inspiration through Observation: Demonstrating the Influence of Automatically Generated Text on Creative Writingcited by 14
  11. 2021Variable-Length Music Score Infilling via XLNet and Musically Specialized Positional Encodingcited by 14
  12. 2023FiLM: Fill-in Language Models for Any-Order Generationcited by 13
31 more
  1. 2024TrajGPT: Controlled Synthetic Trajectory Generation Using a Multitask Transformer-Based Spatiotemporal Modelcited by 13
  2. 2021Machine-in-the-Loop Rewriting for Creative Image Captioningcited by 11
  3. 2022GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentationcited by 11
  4. 2024BEAR: A Unified Framework for Evaluating Relational Knowledge in Causal and Masked Language Modelscited by 11
  5. 2022Crystal Transformer: Self-learning neural language model for Generative and Tinkering Design of Materialscited by 8
  6. 2022Probabilistic Generative Transformer Language models for Generative Design of Moleculescited by 8
  7. 2025On the Impact of Noise in Differentially Private Text Rewritingcited by 8
  8. 2021Language Model Augmented Relevance Scorecited by 7
  9. 2023Toward Unified Controllable Text Generation via Regular Expression Instructioncited by 7
  10. 2021Nutri-bullets: Summarizing Health Studies by Composing Segmentscited by 6
  11. 2022Chinese Idiom Paraphrasingcited by 5
  12. 2023Discovery of 2D materials using Transformer Network based Generative Designcited by 5
  13. 2025MAGNET: Augmenting Generative Decoders with Representation Learning and Infilling Capabilitiescited by 5
  14. 2021Sentence-Permuted Paragraph Generationcited by 4
  15. 2021Nutribullets Hybrid: Multi-document Health Summarizationcited by 3
  16. 2022Controllable Fake Document Infilling for Cyber Deceptioncited by 3
  17. 2022Replacing Language Model for Style Transfercited by 3
  18. 2023Self-Infilling Code Generationcited by 3
  19. 2023Interactive Text Generationcited by 2
  20. 2023Interactive Editing for Text Summarizationcited by 2
  21. 2025Optimal word order for non-causal text generation with Large Language Models: the Spanish casecited by 2
  22. 2026Variational Learning for Insertion-based Generationcited by 2
  23. 2022Materials Transformers Language Models for Generative Materials Design: a benchmark studycited by 1
  24. 2024Restoring Ancient Ideograph: A Multimodal Multitask Neural Network Approachcited by 1
  25. 2021Solving ESL Sentence Completion Questions via Pre-trained Neural Language Modelscited by 0
  26. 2022SC-Ques: A Sentence Completion Question Dataset for English as a Second Language Learnerscited by 0
  27. 2023Unsupervised Text Style Transfer with Deep Generative Modelscited by 0
  28. 2024Towards Probabilistically-Sound Beam Search with Masked Language Modelscited by 0
  29. 2024Predictability and Causality in Spanish and English Natural Language Generationcited by 0
  30. 2026Text Restoration of Ancient Documents with Language Modelscited by 0
  31. 2026Evaluating Whether LLMs Can Reliably Connect the DOTs?cited by 0