# Interview-Informed Generative Agents for Product Discovery: A Validation Study

arXiv: https://arxiv.org/abs/2603.29890 · HTML: https://mlsanity.com/paper/2603.29890/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 20 of its 59 references that are in the map.
- First posted on arXiv 10 Mar 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 20 of 20)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024)
- [Large language models that replace human participants can harmfully misportray and flatten identity groups](https://mlsanity.com/paper/2402.01908/) (2024)
- [From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents](https://mlsanity.com/paper/2412.03563/) (2024)
- [Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions](https://mlsanity.com/paper/2502.16761/) (2025)
- [The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference Perspective](https://mlsanity.com/paper/2312.15524/) (2023)
- [UXAgent: An LLM Agent-Based Usability Testing Framework for Web Design](https://mlsanity.com/paper/2502.12561/) (2025)
- ['Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participants](https://mlsanity.com/paper/2409.19430/) (2024)
- [LLMs Simulate Big Five Personality Traits: Further Evidence](https://mlsanity.com/paper/2402.01765/) (2024)
- [Large Language Models Do Not Simulate Human Psychology](https://mlsanity.com/paper/2508.06950/) (2025)
- [Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions](https://mlsanity.com/paper/1905.10700/) (2019)
- [Generative AI in Knowledge Work: Design Implications for Data Navigation and Decision-Making](https://mlsanity.com/paper/2503.18419/) (2025)
- [Approximating Human Strategic Reasoning with LLM-Enhanced Recursive Reasoners Leveraging Multi-agent Hypergames](https://mlsanity.com/paper/2502.07443/) (2025)
- [Understanding Questions that Arise When Working with Business Documents](https://mlsanity.com/paper/2203.15073/) (2022)
- [Supporting Business Document Workflows via Collection-Centric Information Foraging with Large Language Models](https://mlsanity.com/paper/2405.01501/) (2024)
- [Are Human Interactions Replicable by Generative Agents? A Case Study on Pronoun Usage in Hierarchical Interactions](https://mlsanity.com/paper/2501.15283/) (2025)

## Built on by (most cited first, 1 of 1)

- [Training Computer Use Agents to Assess the Usability of Graphical User Interfaces](https://mlsanity.com/paper/2604.26020/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have shown strong performance on standardized social science instruments, but their value for product discovery remains unclear. We investigate whether interview-informed generative agents can simulate user responses in concept testing scenarios. Using in-depth workflow interviews with knowledge workers, we created personalized agents and compared their evaluations of novel AI concepts against the same participants' responses. Our results show that agents are distribution-calibrated but identity-imprecise: they fail to replicate the specific individual they are grounded in, yet approximate population-level response distributions. These findings highlight both the potential and the limits of LLM simulation in design research. While unsuitable as a substitute for individual-level insights, simulation may provide value for early-stage concept screening and iteration, where distributional accuracy suffices. We discuss implications for integrating simulation responsibly into product development workflows.