BERTian Poetics: Constrained Composition with Masked LMs
Not on the map. Why?
Abstract (arXiv)
Masked language models have recently been interpreted as energy-based sequence models that can be generated from using a Metropolis--Hastings sampler. This short paper demonstrates how this can be instrumentalized for constrained composition and explores the poetics implied by such a usage. Our focus on constraints makes it especially apt to understand the generated text through the poetics of the OuLiPo movement.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 23 references that are in the map. First posted on arXiv 28 Oct 2021.
Builds on
6 of 23 references.
References · 6
- 2017Attention Is All You Needcited by 60,361
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2021Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis--Hastingscited by 37
- 2018Deep-speare: A Joint Neural Model of Poetic Language, Meter and Rhymecited by 35
Built on by
Not cited yet.