# From Lived Experience to Insight: Unpacking the Psychological Risks of Using AI Conversational Agents

arXiv: https://arxiv.org/abs/2412.07951 · HTML: https://mlsanity.com/paper/2412.07951/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 136 references that are in the map.
- First posted on arXiv 10 Dec 2024.

## Builds on (most cited first, 30 of 41)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://mlsanity.com/paper/2307.04657/) (2023)
- [Language Models are Multilingual Chain-of-Thought Reasoners](https://mlsanity.com/paper/2210.03057/) (2022)
- [Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment](https://mlsanity.com/paper/2308.05374/) (2023)
- [Toxicity in ChatGPT: Analyzing Persona-assigned Language Models](https://mlsanity.com/paper/2304.05335/) (2023)
- [LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models](https://mlsanity.com/paper/2308.11462/) (2023)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale](https://mlsanity.com/paper/2211.03759/) (2022)
- [Stable Bias: Analyzing Societal Representations in Diffusion Models](https://mlsanity.com/paper/2303.11408/) (2023)
- [AI Deception: A Survey of Examples, Risks, and Potential Solutions](https://mlsanity.com/paper/2308.14752/) (2023)
- [Gender bias and stereotypes in Large Language Models](https://mlsanity.com/paper/2308.14921/) (2023)
- [A Roadmap to Pluralistic Alignment](https://mlsanity.com/paper/2402.05070/) (2024)
- [Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy](https://mlsanity.com/paper/2002.04087/) (2020)
- [LAMOL: LAnguage MOdeling for Lifelong Language Learning](https://mlsanity.com/paper/1909.03329/) (2019)
- [Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction](https://mlsanity.com/paper/2210.05791/) (2022)
- [Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals](https://mlsanity.com/paper/2210.01790/) (2022)
- [Explanations Can Reduce Overreliance on AI Systems During Decision-Making](https://mlsanity.com/paper/2212.06823/) (2022)
- [The Ethics of Advanced AI Assistants](https://mlsanity.com/paper/2404.16244/) (2024)

## Built on by (most cited first, 25 of 25)

- [ELEPHANT: Measuring and understanding social sycophancy in LLMs](https://mlsanity.com/paper/2505.13995/) (2025)
- [Measuring Agents in Production](https://mlsanity.com/paper/2512.04123/) (2025)
- [Characterizing Delusional Spirals through Human-LLM Chat Logs](https://mlsanity.com/paper/2603.16567/) (2026)
- [Dehumanizing Machines: Mitigating Anthropomorphic Behaviors in Text Generation Systems](https://mlsanity.com/paper/2502.14019/) (2025)
- [Longitudinal Study on Social and Emotional Use of AI Conversational Agent](https://mlsanity.com/paper/2504.14112/) (2025)
- [RubRIX: Rubric-Driven Risk Mitigation in Caregiver-AI Interactions](https://mlsanity.com/paper/2601.13235/) (2026)
- [Mental Health Impacts of AI Companions: Triangulating Social Media Quasi-Experiments, User Perspectives, and Relational Theory](https://mlsanity.com/paper/2509.22505/) (2025)
- [PAIR-SAFE: A Paired-Agent Approach for Runtime Auditing and Refining AI-Mediated Mental Health Support](https://mlsanity.com/paper/2601.12754/) (2026)
- [A Risk Ontology for Evaluating AI-Powered Psychotherapy Virtual Agents](https://mlsanity.com/paper/2505.15108/) (2025)
- [AI Chatbot Suicide Risk Detection and Response: Human Validation Study of the Open-Source VERA-MH Safety Evaluation](https://mlsanity.com/paper/2602.05088/) (2026)
- [When AI Gives Advice: Evaluating AI and Human Responses to Online Advice-Seeking for Well-Being](https://mlsanity.com/paper/2512.08937/) (2025)
- [Assessing Risks of Large Language Models in Mental Health Support: A Framework for Automated Clinical AI Red Teaming](https://mlsanity.com/paper/2602.19948/) (2026)
- [MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Models](https://mlsanity.com/paper/2604.17730/) (2026)
- [Seeking Late Night Life Lines: Experiences of Conversational AI Use in Mental Health Crisis](https://mlsanity.com/paper/2512.23859/) (2025)
- [Brainrot: Deskilling and Addiction are Overlooked AI Risks](https://mlsanity.com/paper/2605.03512/) (2026)
- [LLUMI: Improving LLM Writing Assistance for Mental Health Support with Online Community Feedback](https://mlsanity.com/paper/2605.30273/) (2026)
- [AI Chaperones Are (Really) All You Need to Prevent Parasocial Relationships with Chatbots](https://mlsanity.com/paper/2508.15748/) (2025)
- [Lived Experience in Dialogue: Co-designing Personalization in Large Language Models to Support Youth Mental Well-being](https://mlsanity.com/paper/2511.05769/) (2025)
- [Breakdowns in Conversational AI: Interactional Failures in Emotionally and Ethically Sensitive Contexts](https://mlsanity.com/paper/2604.02713/) (2026)
- [EthicMind: A Risk-Aware Framework for Ethical-Emotional Alignment in Multi-Turn Dialogue](https://mlsanity.com/paper/2604.09265/) (2026)
- [Understanding How International Students in the U.S. Are Using Conversational AI to Support Cross-Cultural Adaptation](https://mlsanity.com/paper/2605.15127/) (2026)
- [Examining Human-Like Behaviors in LLMs: A Multi-Dimensional Analysis of Model Behaviors, User Factors, and System Prompts](https://mlsanity.com/paper/2606.18258/) (2026)
- [Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Being](https://mlsanity.com/paper/2607.25057/) (2026)
- [PeerPen: AI-Assisted Writing for Online Mental Health Peer Support](https://mlsanity.com/paper/2609.14886/) (2026)
- [Aligning with Lived Experience: Heterogeneous Benefits of Fine Tuning in Mental Health Support Generation](https://mlsanity.com/paper/2609.21075/) (2026)

## Abstract (arXiv)

Recent gains in popularity of AI conversational agents have led to their increased use for improving productivity and supporting well-being. While previous research has aimed to understand the risks associated with interactions with AI conversational agents, these studies often fall short in capturing the lived experiences of individuals. Additionally, psychological risks have often been presented as a sub-category within broader AI-related risks in past taxonomy works, leading to under-representation of the impact of psychological risks of AI use. To address these challenges, our work presents a novel risk taxonomy focusing on psychological risks of using AI gathered through the lived experiences of individuals. We employed a mixed-method approach, involving a comprehensive survey with 283 people with lived mental health experience and workshops involving experts with lived experience to develop a psychological risk taxonomy. Our taxonomy features 19 AI behaviors, 21 negative psychological impacts, and 15 contexts related to individuals. Additionally, we propose a novel multi-path vignette-based framework for understanding the complex interplay between AI behaviors, psychological impacts, and individual user contexts. Finally, based on the feedback obtained from the workshop sessions, we present design recommendations for developing safer and more robust AI agents. Our work offers an in-depth understanding of the psychological risks associated with AI conversational agents and provides actionable recommendations for policymakers, researchers, and developers.