# Say What? Collaborative Pop Lyric Generation Using Multitask Transfer Learning

arXiv: https://arxiv.org/abs/2111.07592 · HTML: https://mlsanity.com/paper/2111.07592/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 15 Nov 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Exploring the Limits of Language Modeling](https://mlsanity.com/paper/1602.02410/) (2016)
- [Data Augmentation using Pre-trained Transformer Models](https://mlsanity.com/paper/2003.02245/) (2020)
- [Long Text Generation via Adversarial Training with Leaked Information](https://mlsanity.com/paper/1709.08624/) (2017)
- [Tensor2Tensor for Neural Machine Translation](https://mlsanity.com/paper/1803.07416/) (2018)
- [Adversarial Feature Matching for Text Generation](https://mlsanity.com/paper/1706.03850/) (2017)
- [Deep-speare: A Joint Neural Model of Poetic Language, Meter and Rhyme](https://mlsanity.com/paper/1807.03491/) (2018)
- [DopeLearning: A Computational Approach to Rap Lyrics Generation](https://mlsanity.com/paper/1505.04771/) (2015)
- [Natural Language Statistical Features of LSTM-generated Texts](https://mlsanity.com/paper/1804.04087/) (2018)
- [Shimon the Rapper: A Real-Time System for Human-Robot Interactive Rap Battles](https://mlsanity.com/paper/2009.09234/) (2020)
- [Weird AI Yankovic: Generating Parody Lyrics](https://mlsanity.com/paper/2009.12240/) (2020)
- [Rapformer: Conditional Rap Lyrics Generation with Denoising Autoencoders](https://mlsanity.com/paper/2004.03965/) (2020)

## Built on by (most cited first, 7 of 7)

- [Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process](https://mlsanity.com/paper/2504.12488/) (2025)
- [REFFLY: Melody-Constrained Lyrics Editing Model](https://mlsanity.com/paper/2409.00292/) (2024)
- [Automated Evaluation of Meter and Rhyme in Russian Generative and Human-Authored Poetry](https://mlsanity.com/paper/2502.20931/) (2025)
- [CoLyricist: Enhancing Lyric Writing with AI through Workflow-Aligned Support](https://mlsanity.com/paper/2602.22606/) (2026)
- [Controllable Ancient Chinese Lyrics Generation Based on Phrase Prototype Retrieving](https://mlsanity.com/paper/2303.11005/) (2023)
- [Encoder-Decoder Framework for Interactive Free Verses with Generation with Controllable High-Quality Rhyming](https://mlsanity.com/paper/2405.05176/) (2024)
- [Detecting Spelling and Grammatical Anomalies in Russian Poetry Texts](https://mlsanity.com/paper/2505.04507/) (2025)

## Abstract (arXiv)

Lyric generation is a popular sub-field of natural language generation that has seen growth in recent years. Pop lyrics are of unique interest due to the genre's unique style and content, in addition to the high level of collaboration that goes on behind the scenes in the professional pop songwriting process. In this paper, we present a collaborative line-level lyric generation system that utilizes transfer-learning via the T5 transformer model, which, till date, has not been used to generate pop lyrics. By working and communicating directly with professional songwriters, we develop a model that is able to learn lyrical and stylistic tasks like rhyming, matching line beat requirements, and ending lines with specific target words. Our approach compares favorably to existing methods for multiple datasets and yields positive results from our online studies and interviews with industry songwriters.