mlsanityData to 7 Oct 2026

Sparks: Inspiration for Science Writing using Language Models

arXiv 2110.07640 ↗First posted 14 Oct 2021cs.HC cs.CLcited by 80

Not on the map. Why?

Abstract (arXiv)

Large-scale language models are rapidly improving, performing well on a wide variety of tasks with little to no customization. In this work we investigate how language models can support science writing, a challenging writing task that is both open-ended and highly constrained. We present a system for generating "sparks", sentences related to a scientific concept intended to inspire writers. We find that our sparks are more coherent and diverse than a competitive language model baseline, and approach a human-created gold standard. In a study with 13 PhD students writing on topics of their own selection, we find three main use cases of sparks: aiding with crafting detailed sentences, providing interesting angles to engage readers, and demonstrating common reader perspectives. We also report on the various reasons sparks were considered unhelpful, and discuss how we might improve language models as writing support tools.

Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). Builds on 30 of its 86 references that are in the map. First posted on arXiv 14 Oct 2021.

Builds on

30 of 86 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  8. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  9. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  10. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  11. 2018Hierarchical Neural Story Generationcited by 1,182
  12. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
18 more
  1. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  2. 2021GPT Understands, Toocited by 715
  3. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  4. 2020Towards a Human-like Open-Domain Chatbotcited by 467
  5. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 299
  6. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  7. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 241
  8. 2020If beam search is the answer, what was the question?cited by 114
  9. 2020CTRLsum: Towards Generic Controllable Text Summarizationcited by 98
  10. 2019Gmail Smart Compose: Real-Time Assisted Writingcited by 97
  11. 2019Comparison of Diverse Decoding Methods from Conditional Language Modelscited by 78
  12. 2016Smart Reply: Automated Response Suggestion for Emailcited by 54
  13. 2021MERMAID: Metaphor Generation with Symbolism and Discriminative Decodingcited by 49
  14. 2021Addressing "Documentation Debt" in Machine Learning Research: A Retrospective Datasheet for BookCorpuscited by 46
  15. 2021Wordcraft: a Human-AI Collaborative Editor for Story Writingcited by 41
  16. 2021CharacterChat: Supporting the Creation of Fictional Characters through Conversation and Progressive Manifestation with a Chatbotcited by 18
  17. 2018Domain Agnostic Real-Valued Specificity Predictioncited by 10
  18. 2020The COVID-19 Infodemic: Twitter versus Facebookcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 28; 2024: 25; 2025: 13; 2026: 9.

80 papers.

Sort:

By area:

  1. 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
  2. 2023Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrieverscited by 91
  3. 2024A Design Space for Intelligent and Interactive Writing Assistantscited by 76
  4. 2024Building Machines that Learn and Think with Peoplecited by 73
  5. 2022Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Supportcited by 68
  6. 2023Graphologue: Exploring Large Language Model Responses with Interactive Diagramscited by 65
  7. 2023Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creationcited by 63
  8. 2024AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuancescited by 59
  9. 2023On the Creativity of Large Language Modelscited by 57
  10. 2024Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Modelscited by 51
  11. 2023MindfulDiary: Harnessing Large Language Model to Support Psychiatric Patients' Journalingcited by 50
  12. 2024Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Editscited by 49
38 more
  1. 2022Read, Revise, Repeat: A System Demonstration for Human-in-the-loop Iterative Text Revisioncited by 46
  2. 2023VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototypingcited by 46
  3. 2024M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detectioncited by 43
  4. 20223DALL-E: Integrating Text-to-Image AI in 3D Design Workflowscited by 41
  5. 2023Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Promptingcited by 36
  6. 2023Beyond the Chat: Executable and Verifiable Text-Editing with LLMscited by 35
  7. 2023ConstitutionMaker: Interactively Critiquing Large Language Models by Converting Feedback into Principlescited by 33
  8. 2023Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writerscited by 32
  9. 2024SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detectioncited by 30
  10. 2024The Silicon Ceiling: Auditing GPT's Race and Gender Biases in Hiringcited by 29
  11. 2024A Robot Walks into a Bar: Can Language Models Serve as Creativity Support Tools for Comedy? An Evaluation of LLMs' Humour Alignment with Comedianscited by 29
  12. 2023DataTales: Investigating the use of Large Language Models for Authoring Data-Driven Articlescited by 24
  13. 2025AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computationcited by 24
  14. 2022Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processingcited by 23
  15. 2023Fluid Transformers and Creative Analogies: Exploring Large Language Models' Capacity for Augmenting Cross-Domain Analogical Creativitycited by 23
  16. 2024How AI Ideas Affect the Creativity, Diversity, and Evolution of Human Ideas: Evidence From a Large, Dynamic Experimentcited by 22
  17. 2023Beyond Summarization: Designing AI Support for Real-World Expository Writing Taskscited by 20
  18. 2023Generative Disco: Text-to-Video Generation for Music Visualizationcited by 19
  19. 2023ReelFramer: Human-AI Co-Creation for News-to-Video Translationcited by 19
  20. 2023ABScribe: Rapid Exploration & Organization of Multiple Writing Variations in Human-AI Co-Writing Tasks using Large Language Modelscited by 17
  21. 2023"It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Modelscited by 15
  22. 2023Unraveling Downstream Gender Bias from Large Language Models: A Study on AI Educational Writing Assistancecited by 15
  23. 2024"It was 80% me, 20% AI": Seeking Authenticity in Co-Writing with Large Language Modelscited by 14
  24. 2025Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Processcited by 14
  25. 2025SurGE: A Benchmark and Evaluation Framework for Scientific Survey Generationcited by 14
  26. 2024Not Just Novelty: A Longitudinal Study on Utility and Customization of an AI Workflowcited by 13
  27. 2024Authors' Values and Attitudes Towards AI-bridged Scalable Personalization of Creative Language Artscited by 13
  28. 2024Collage is the New Writing: Exploring the Fragmentation of Text and User Interfaces in AI Toolscited by 13
  29. 2023Tweetorial Hooks: Generative AI Tools to Motivate Science on Social Mediacited by 11
  30. 2023Exploring Perspectives on the Impact of Artificial Intelligence on the Creativity of Knowledge Work: Beyond Mechanised Plagiarism and Stochastic Parrotscited by 11
  31. 2023SmartBook: AI-Assisted Situation Report Generation for Intelligence Analystscited by 9
  32. 2024The Dearth of the Author in AI-Supported Writingcited by 9
  33. 2026Can Good Writing Be Generative? Expert-Level AI Writing Emerges through Fine-Tuning on High-Quality Bookscited by 9
  34. 2023PromptInfuser: How Tightly Coupling AI and UI Design Impacts Designers' Workflowscited by 8
  35. 2024SARD: A Human-AI Collaborative Story Generationcited by 8
  36. 2024DOLOMITES: Domain-Specific Long-Form Methodical Taskscited by 8
  37. 2025Where Do I 'Add the Egg'?: Exploring Agency and Ownership in AI Creative Co-Writing Systemscited by 7
  38. 2023Mapping the Design Space of Interactions in Human-AI Text Co-creation Taskscited by 6

50 most cited of 80 All 80 (JSON)