# ChatGPTest: opportunities and cautionary tales of utilizing AI for questionnaire pretesting

arXiv: https://arxiv.org/abs/2405.06329 · HTML: https://mlsanity.com/paper/2405.06329/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 22 references that are in the map.
- First posted on arXiv 10 May 2024.

## Builds on (most cited first, 3 of 3)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)

## Built on by (most cited first, 1 of 1)

- [Assessing instructor-AI cooperation for grading essay-type questions in an introductory sociology course](https://mlsanity.com/paper/2501.06461/) (2025)

## Abstract (arXiv)

The rapid advancements in generative artificial intelligence have opened up new avenues for enhancing various aspects of research, including the design and evaluation of survey questionnaires. However, the recent pioneering applications have not considered questionnaire pretesting. This article explores the use of GPT models as a useful tool for pretesting survey questionnaires, particularly in the early stages of survey design. Illustrated with two applications, the article suggests incorporating GPT feedback as an additional stage before human pretesting, potentially reducing successive iterations. The article also emphasizes the indispensable role of researchers' judgment in interpreting and implementing AI-generated feedback.