# AI Revolution on Chat Bot: Evidence from a Randomized Controlled Experiment

arXiv: https://arxiv.org/abs/2401.10956 · HTML: https://mlsanity.com/paper/2401.10956/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 8 references that are in the map.
- First posted on arXiv 19 Jan 2024.

## Builds on (most cited first, 2 of 2)

- [The Impact of AI on Developer Productivity: Evidence from GitHub Copilot](https://mlsanity.com/paper/2302.06590/) (2023)
- [Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized Experiment](https://mlsanity.com/paper/2307.03744/) (2023)

## Built on by (most cited first, 1 of 1)

- [Prompt Recursive Search: A Living Framework with Adaptive Growth in LLM Auto-Prompting](https://mlsanity.com/paper/2408.01423/) (2024)

## Abstract (arXiv)

In recent years, generative AI has undergone major advancements, demonstrating significant promise in augmenting human productivity. Notably, large language models (LLM), with ChatGPT-4 as an example, have drawn considerable attention. Numerous articles have examined the impact of LLM-based tools on human productivity in lab settings and designed tasks or in observational studies. Despite recent advances, field experiments applying LLM-based tools in realistic settings are limited. This paper presents the findings of a field randomized controlled trial assessing the effectiveness of LLM-based tools in providing unmonitored support services for information retrieval.