# Power-up! What Can Generative Models Do for Human Computation Workflows?

arXiv: https://arxiv.org/abs/2307.02243 · HTML: https://mlsanity.com/paper/2307.02243/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 49 references that are in the map.
- First posted on arXiv 5 Jul 2023.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [COMET: Commonsense Transformers for Automatic Knowledge Graph Construction](https://mlsanity.com/paper/1906.05317/) (2019)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Explain Yourself! Leveraging Language Models for Commonsense Reasoning](https://mlsanity.com/paper/1906.02361/) (2019)
- [Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models](https://mlsanity.com/paper/2101.00288/) (2021)
- [WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation](https://mlsanity.com/paper/2201.05955/) (2022)
- [FinBERT: A Pretrained Language Model for Financial Communications](https://mlsanity.com/paper/2006.08097/) (2020)
- [Calibrating Factual Knowledge in Pretrained Language Models](https://mlsanity.com/paper/2210.03329/) (2022)
- [Sparks: Inspiration for Science Writing using Language Models](https://mlsanity.com/paper/2110.07640/) (2021)
- [DocBERT: BERT for Document Classification](https://mlsanity.com/paper/1904.08398/) (2019)
- [Patent Claim Generation by Fine-Tuning OpenAI GPT-2](https://mlsanity.com/paper/1907.02052/) (2019)
- [On the Explainability of Natural Language Processing Deep Models](https://mlsanity.com/paper/2210.06929/) (2022)
- [Learning-to-Rank with BERT in TF-Ranking](https://mlsanity.com/paper/2004.08476/) (2020)
- [Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants](https://mlsanity.com/paper/2112.09062/) (2021)
- [On the Compositional Generalization Gap of In-Context Learning](https://mlsanity.com/paper/2211.08473/) (2022)
- [Towards Process-Oriented, Modular, and Versatile Question Generation that Meets Educational Needs](https://mlsanity.com/paper/2205.00355/) (2022)

## Built on by (most cited first, 1 of 1)

- [Designing LLM Chains by Adapting Techniques from Crowdsourcing Workflows](https://mlsanity.com/paper/2312.11681/) (2023)

## Abstract (arXiv)

We are amidst an explosion of artificial intelligence research, particularly around large language models (LLMs). These models have a range of applications across domains like medicine, finance, commonsense knowledge graphs, and crowdsourcing. Investigation into LLMs as part of crowdsourcing workflows remains an under-explored space. The crowdsourcing research community has produced a body of work investigating workflows and methods for managing complex tasks using hybrid human-AI methods. Within crowdsourcing, the role of LLMs can be envisioned as akin to a cog in a larger wheel of workflows. From an empirical standpoint, little is currently understood about how LLMs can improve the effectiveness of crowdsourcing workflows and how such workflows can be evaluated. In this work, we present a vision for exploring this gap from the perspectives of various stakeholders involved in the crowdsourcing paradigm -- the task requesters, crowd workers, platforms, and end-users. We identify junctures in typical crowdsourcing workflows at which the introduction of LLMs can play a beneficial role and propose means to augment existing design patterns for crowd work.