# Exploring Conversational Agents as an Effective Tool for Measuring Cognitive Biases in Decision-Making

arXiv: https://arxiv.org/abs/2401.06686 · HTML: https://mlsanity.com/paper/2401.06686/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 28 references that are in the map.
- First posted on arXiv 8 Jan 2024.

## Builds on (most cited first, 6 of 6)

- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Frames: A Corpus for Adding Memory to Goal-Oriented Dialogue Systems](https://mlsanity.com/paper/1704.00057/) (2017)
- [Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data](https://mlsanity.com/paper/2301.05843/) (2023)
- [A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions](https://mlsanity.com/paper/1906.00500/) (2019)
- [Stylized Dialogue Response Generation Using Stylized Unpaired Texts](https://mlsanity.com/paper/2009.12719/) (2020)
- [BIASeD: Bringing Irrationality into Automated System Design](https://mlsanity.com/paper/2210.01122/) (2022)

## Built on by (most cited first, 4 of 4)

- [A Comprehensive Evaluation of Cognitive Biases in LLMs](https://mlsanity.com/paper/2410.15413/) (2024)
- [DERELAB: Probing Defeasible Reasoning and Confirmation Bias in LLMs with a Generative Benchmark](https://mlsanity.com/paper/2608.30413/) (2026)
- [Predicting Biased Human Decision-Making with Large Language Models in Conversational Settings](https://mlsanity.com/paper/2601.11049/) (2026)
- [Emulating Aggregate Human Choice Behavior and Biases with GPT Conversational Agents](https://mlsanity.com/paper/2602.05597/) (2026)

## Abstract (arXiv)

Heuristics and cognitive biases are an integral part of human decision-making. Automatically detecting a particular cognitive bias could enable intelligent tools to provide better decision-support. Detecting the presence of a cognitive bias currently requires a hand-crafted experiment and human interpretation. Our research aims to explore conversational agents as an effective tool to measure various cognitive biases in different domains. Our proposed conversational agent incorporates a bias measurement mechanism that is informed by the existing experimental designs and various experimental tasks identified in the literature. Our initial experiments to measure framing and loss-aversion biases indicate that the conversational agents can be effectively used to measure the biases.