# ChatGPT and U(X): A Rapid Review on Measuring the User Experience

arXiv: https://arxiv.org/abs/2503.15808 · HTML: https://mlsanity.com/paper/2503.15808/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 129 references that are in the map.
- First posted on arXiv 20 Mar 2025.

## Builds on (most cited first, 22 of 22)

- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [Co-Writing with Opinionated Language Models Affects Users' Views](https://mlsanity.com/paper/2302.00560/) (2023)
- [DirectGPT: A Direct Manipulation Interface to Interact with Large Language Models](https://mlsanity.com/paper/2310.03691/) (2023)
- [Why and When LLM-Based Assistants Can Go Wrong: Investigating the Effectiveness of Prompt-Based Interactions for Software Help-Seeking](https://mlsanity.com/paper/2402.08030/) (2024)
- [The State of Speech in HCI: Trends, Themes and Challenges](https://mlsanity.com/paper/1810.06828/) (2018)
- [CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language Models](https://mlsanity.com/paper/2402.15265/) (2024)
- [Improved Trust in Human-Robot Collaboration with ChatGPT](https://mlsanity.com/paper/2304.12529/) (2023)
- [Rambler: Supporting Writing With Speech via LLM-Assisted Gist Manipulation](https://mlsanity.com/paper/2401.10838/) (2024)
- [Let's have a chat! A Conversation with ChatGPT: Technology, Applications, and Limitations](https://mlsanity.com/paper/2302.13817/) (2023)
- [Nonverbal Cues in Human-Robot Interaction: A Communication Studies Perspective](https://mlsanity.com/paper/2304.11293/) (2023)
- [User Interaction Patterns and Breakdowns in Conversing with LLM-Powered Voice Assistants](https://mlsanity.com/paper/2309.13879/) (2023)
- [HILL: A Hallucination Identifier for Large Language Models](https://mlsanity.com/paper/2403.06710/) (2024)
- [Closing the Loop: Testing ChatGPT to Generate Model Explanations to Improve Human Labelling of Sponsored Content on Social Media](https://mlsanity.com/paper/2306.05115/) (2023)
- [Measuring Voice UX Quantitatively: A Rapid Review](https://mlsanity.com/paper/2103.07108/) (2021)
- [CUI@CHI 2024: Building Trust in CUIs-From Design to Deployment](https://mlsanity.com/paper/2401.13970/) (2024)
- [Silver-Tongued and Sundry: Exploring Intersectional Pronouns with ChatGPT](https://mlsanity.com/paper/2405.08238/) (2024)
- [User Centric Evaluation of Code Generation Tools](https://mlsanity.com/paper/2402.03130/) (2024)
- [Enhancing Programming Error Messages in Real Time with Generative AI](https://mlsanity.com/paper/2402.08072/) (2024)
- [The Systematic Review-lution: A Manifesto to Promote Rigour and Inclusivity in Research Synthesis](https://mlsanity.com/paper/2304.13556/) (2023)
- [The Partner Modelling Questionnaire: A validated self-report measure of perceptions toward machines as dialogue partners](https://mlsanity.com/paper/2308.07164/) (2023)
- [Unraveling the Dilemma of AI Errors: Exploring the Effectiveness of Human and Machine Explanations for Large Language Models](https://mlsanity.com/paper/2404.07725/) (2024)
- [The Impact of Performance Expectancy, Workload, Risk, and Satisfaction on Trust in ChatGPT: Cross-sectional Survey Analysis](https://mlsanity.com/paper/2311.05632/) (2023)

## Built on by (most cited first, 1 of 1)

- [Digital Companionship: Overlapping Uses of AI Companions and AI Assistants](https://mlsanity.com/paper/2510.15905/) (2025)

## Abstract (arXiv)

ChatGPT, powered by a large language model (LLM), has revolutionized everyday human-computer interaction (HCI) since its 2022 release. While now used by millions around the world, a coherent pathway for evaluating the user experience (UX) ChatGPT offers remains missing. In this rapid review (N = 58), I explored how ChatGPT UX has been approached quantitatively so far. I focused on the independent variables (IVs) manipulated, the dependent variables (DVs) measured, and the methods used for measurement. Findings reveal trends, gaps, and emerging consensus in UX assessments. This work offers a first step towards synthesizing existing approaches to measuring ChatGPT UX, urgent trajectories to advance standardization and breadth, and two preliminary frameworks aimed at guiding future research and tool development. I seek to elevate the field of ChatGPT UX by empowering researchers and practitioners in optimizing user interactions with ChatGPT and similar LLM-based systems.