# Redefining Research Crowdsourcing: Incorporating Human Feedback with LLM-Powered Digital Twins

arXiv: https://arxiv.org/abs/2505.24004 · HTML: https://mlsanity.com/paper/2505.24004/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 32 references that are in the map.
- First posted on arXiv 29 May 2025.

## Builds on (most cited first, 8 of 8)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals](https://mlsanity.com/paper/2411.10109/) (2024)
- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks](https://mlsanity.com/paper/2306.07899/) (2023)
- [LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMs](https://mlsanity.com/paper/2307.10168/) (2023)
- [Interpretable Directed Diversity: Leveraging Model Explanations for Iterative Crowd Ideation](https://mlsanity.com/paper/2109.10149/) (2021)

## Built on by (most cited first, 3 of 3)

- [Discovering Differences in Strategic Behavior Between Humans and LLMs](https://mlsanity.com/paper/2602.10324/) (2026)
- [Item-Mean Surrogates: Why Richer Persona Data Fail to Improve LLMs as Human Surrogates](https://mlsanity.com/paper/2608.29455/) (2026)
- [Lived Experience in Dialogue: Co-designing Personalization in Large Language Models to Support Youth Mental Well-being](https://mlsanity.com/paper/2511.05769/) (2025)

## Abstract (arXiv)

Crowd work platforms like Amazon Mechanical Turk and Prolific are vital for research, yet workers' growing use of generative AI tools poses challenges. Researchers face compromised data validity as AI responses replace authentic human behavior, while workers risk diminished roles as AI automates tasks. To address this, we propose a hybrid framework using digital twins, personalized AI models that emulate workers' behaviors and preferences while keeping humans in the loop. We evaluate our system with an experiment (n=88 crowd workers) and in-depth interviews with crowd workers (n=5) and social science researchers (n=4). Our results suggest that digital twins may enhance productivity and reduce decision fatigue while maintaining response quality. Both researchers and workers emphasized the importance of transparency, ethical data use, and worker agency. By automating repetitive tasks and preserving human engagement for nuanced ones, digital twins may help balance scalability with authenticity.