# What does ChatGPT return about human values? Exploring value bias in ChatGPT using a descriptive value theory

arXiv: https://arxiv.org/abs/2304.03612 · HTML: https://mlsanity.com/paper/2304.03612/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 39 references that are in the map.
- First posted on arXiv 7 Apr 2023.

## Builds on (most cited first, 7 of 7)

- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Probing Pre-Trained Language Models for Cross-Cultural Differences in Values](https://mlsanity.com/paper/2203.13722/) (2022)
- [Harms from Increasingly Agentic Algorithmic Systems](https://mlsanity.com/paper/2302.10329/) (2023)
- [Eight Things to Know about Large Language Models](https://mlsanity.com/paper/2304.00612/) (2023)

## Built on by (most cited first, 19 of 19)

- [A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets](https://mlsanity.com/paper/2305.18486/) (2023)
- [Are Large Language Models Consistent over Value-laden Questions?](https://mlsanity.com/paper/2407.02996/) (2024)
- [Large Language Models Reflect the Ideology of their Creators](https://mlsanity.com/paper/2410.18417/) (2024)
- [Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Values](https://mlsanity.com/paper/2311.10766/) (2023)
- [DOSA: A Dataset of Social Artifacts from Different Indian Geographical Subcultures](https://mlsanity.com/paper/2403.14651/) (2024)
- [Normative Evaluation of Large Language Models with Everyday Moral Dilemmas](https://mlsanity.com/paper/2501.18081/) (2025)
- [Do LLMs have Consistent Values?](https://mlsanity.com/paper/2407.12878/) (2024)
- [How Deep Is Representational Bias in LLMs? The Cases of Caste and Religion](https://mlsanity.com/paper/2508.03712/) (2025)
- [Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models](https://mlsanity.com/paper/2409.12106/) (2024)
- [Knowing But Not Doing: Convergent Morality and Divergent Action in LLMs](https://mlsanity.com/paper/2601.07972/) (2026)
- [GPT-ology, Computational Models, Silicon Sampling: How should we think about LLMs in Cognitive Science?](https://mlsanity.com/paper/2406.09464/) (2024)
- [EAVIT: Efficient and Accurate Human Value Identification from Text data via LLMs](https://mlsanity.com/paper/2505.12792/) (2025)
- [A Group Fairness Lens for Large Language Models](https://mlsanity.com/paper/2312.15478/) (2023)
- [VALUEFLOW: Toward Pluralistic and Steerable Value-based Alignment in Large Language Models](https://mlsanity.com/paper/2602.03160/) (2026)
- [Neural network embeddings recover value dimensions from psychometric survey items on par with human data](https://mlsanity.com/paper/2509.24906/) (2025)
- [Omissive Bias in Religious Representation: Benchmarking LLM Answers to Everyday Ethical Decision-making](https://mlsanity.com/paper/2605.24319/) (2026)
- [Detecting value-expressive text posts in Russian social media](https://mlsanity.com/paper/2312.08968/) (2023)
- [Word Order's Impacts: Insights from Reordering and Generation Analysis](https://mlsanity.com/paper/2403.11473/) (2024)
- [Teaching Values to Machines: Simulating Human-Like Behavior in LLMs](https://mlsanity.com/paper/2605.30036/) (2026)

## Abstract (arXiv)

There has been concern about ideological basis and possible discrimination in text generated by Large Language Models (LLMs). We test possible value biases in ChatGPT using a psychological value theory. We designed a simple experiment in which we used a number of different probes derived from the Schwartz basic value theory (items from the revised Portrait Value Questionnaire, the value type definitions, value names). We prompted ChatGPT via the OpenAI API repeatedly to generate text and then analyzed the generated corpus for value content with a theory-driven value dictionary using a bag of words approach. Overall, we found little evidence of explicit value bias. The results showed sufficient construct and discriminant validity for the generated text in line with the theoretical predictions of the psychological model, which suggests that the value content was carried through into the outputs with high fidelity. We saw some merging of socially oriented values, which may suggest that these values are less clearly differentiated at a linguistic level or alternatively, this mixing may reflect underlying universal human motivations. We outline some possible applications of our findings for both applications of ChatGPT for corporate usage and policy making as well as future research avenues. We also highlight possible implications of this relatively high-fidelity replication of motivational content using a linguistic model for the theorizing about human values.