# Safeguarding Crowdsourcing Surveys from ChatGPT with Prompt Injection

arXiv: https://arxiv.org/abs/2306.08833 · HTML: https://mlsanity.com/paper/2306.08833/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 15 Jun 2023.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 16 of 16)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Ignore Previous Prompt: Attack Techniques For Language Models](https://mlsanity.com/paper/2211.09527/) (2022)
- [AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators](https://mlsanity.com/paper/2303.16854/) (2023)
- [Automatic Detection of Machine Generated Text: A Critical Survey](https://mlsanity.com/paper/2011.01314/) (2020)
- [Does Synthetic Data Generation of LLMs Help Clinical Text Mining?](https://mlsanity.com/paper/2303.04360/) (2023)
- [ChatGPT Makes Medicine Easy to Swallow: An Exploratory Case Study on Simplified Radiology Reports](https://mlsanity.com/paper/2212.14882/) (2022)
- [Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods](https://mlsanity.com/paper/2210.07321/) (2022)
- [Legal Prompt Engineering for Multilingual Legal Judgement Prediction](https://mlsanity.com/paper/2212.02199/) (2022)
- [On the Educational Impact of ChatGPT: Is Artificial Intelligence Ready to Obtain a University Degree?](https://mlsanity.com/paper/2303.11146/) (2023)
- [Putting ChatGPT's Medical Advice to the (Turing) Test](https://mlsanity.com/paper/2301.10035/) (2023)
- [LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework](https://mlsanity.com/paper/2112.07522/) (2021)

## Built on by (most cited first, 11 of 11)

- [Automatic and Universal Prompt Injection Attacks against Large Language Models](https://mlsanity.com/paper/2403.04957/) (2024)
- [System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspective](https://mlsanity.com/paper/2409.19091/) (2024)
- [A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems](https://mlsanity.com/paper/2402.18649/) (2024)
- [InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models](https://mlsanity.com/paper/2410.22770/) (2024)
- [Comprehensive Evaluation of ChatGPT Reliability Through Multilingual Inquiries](https://mlsanity.com/paper/2312.10524/) (2023)
- [Indirect Prompt Injection in the Wild: An Empirical Study of Prevalence, Techniques, and Objectives](https://mlsanity.com/paper/2604.27202/) (2026)
- [JavelinGuard: Low-Cost Transformer Architectures for LLM Security](https://mlsanity.com/paper/2506.07330/) (2025)
- [Prompt-in-Content Attacks: Exploiting Uploaded Inputs to Hijack LLM Behavior](https://mlsanity.com/paper/2508.19287/) (2025)
- [Multi-Agent Framework for Threat Mitigation and Resilience in AI-Based Systems](https://mlsanity.com/paper/2512.23132/) (2025)
- [Software Vulnerability Management in the Era of Artificial Intelligence: An Industry Perspective](https://mlsanity.com/paper/2512.18261/) (2025)
- [Responsible AI for General-Purpose Systems: Overview, Challenges, and A Path Forward](https://mlsanity.com/paper/2601.13122/) (2026)

## Abstract (arXiv)

ChatGPT and other large language models (LLMs) have proven useful in crowdsourcing tasks, where they can effectively annotate machine learning training data. However, this means that they also have the potential for misuse, specifically to automatically answer surveys. LLMs can potentially circumvent quality assurance measures, thereby threatening the integrity of methodologies that rely on crowdsourcing surveys. In this paper, we propose a mechanism to detect LLM-generated responses to surveys. The mechanism uses "prompt injection", such as directions that can mislead LLMs into giving predictable responses. We evaluate our technique against a range of question scenarios, types, and positions, and find that it can reliably detect LLM-generated responses with more than 93% effectiveness. We also provide an open-source software to help survey designers use our technique to detect LLM responses. Our work is a step in ensuring that survey methodologies remain rigorous vis-a-vis LLMs.