# The Dark Side of ChatGPT: Legal and Ethical Challenges from Stochastic Parrots and Hallucination

arXiv: https://arxiv.org/abs/2304.14347 · HTML: https://mlsanity.com/paper/2304.14347/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 13 references that are in the map.
- First posted on arXiv 21 Apr 2023.

## Builds on (most cited first, 5 of 5)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Black-Box Tuning for Language-Model-as-a-Service](https://mlsanity.com/paper/2201.03514/) (2022)
- [Regulating ChatGPT and other Large Generative AI Models](https://mlsanity.com/paper/2302.02337/) (2023)
- [Using sensitive data to prevent discrimination by artificial intelligence: Does the GDPR need a new exception?](https://mlsanity.com/paper/2206.03262/) (2022)
- [Regulating Gatekeeper AI and Data: Transparency, Access, and Fairness under the DMA, the GDPR, and beyond](https://mlsanity.com/paper/2212.04997/) (2022)

## Built on by (most cited first, 23 of 23)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Confabulation: The Surprising Value of Large Language Model Hallucinations](https://mlsanity.com/paper/2406.04175/) (2024)
- [LexEval: A Comprehensive Chinese Legal Benchmark for Evaluating Large Language Models](https://mlsanity.com/paper/2409.20288/) (2024)
- [AI Hallucinations: A Misnomer Worth Clarifying](https://mlsanity.com/paper/2401.06796/) (2024)
- [Chameleon: a Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models](https://mlsanity.com/paper/2310.09949/) (2023)
- [AI-assisted Code Authoring at Scale: Fine-tuning, deploying, and mixed methods evaluation](https://mlsanity.com/paper/2305.12050/) (2023)
- [Mapping the Ethics of Generative AI: A Comprehensive Scoping Review](https://mlsanity.com/paper/2402.08323/) (2024)
- [Do Language Models Mirror Human Confidence? Exploring Psychological Insights to Address Overconfidence in LLMs](https://mlsanity.com/paper/2506.00582/) (2025)
- [The Social Impact of Generative AI: An Analysis on ChatGPT](https://mlsanity.com/paper/2403.04667/) (2024)
- [Towards Reliable Retrieval in RAG Systems for Large Legal Datasets](https://mlsanity.com/paper/2510.06999/) (2025)
- [The Stochastic Parrot on LLM's Shoulder: A Summative Assessment of Physical Concept Understanding](https://mlsanity.com/paper/2502.08946/) (2025)
- [International Governance of Civilian AI: A Jurisdictional Certification Approach](https://mlsanity.com/paper/2308.15514/) (2023)
- [LLM vs. Lawyers: Identifying a Subset of Summary Judgments in a Large UK Case Law Dataset](https://mlsanity.com/paper/2403.04791/) (2024)
- [The Accountability Paradox: How Platform API Restrictions Undermine AI Transparency Mandates](https://mlsanity.com/paper/2505.11577/) (2025)
- [Exploring the Nexus of Large Language Models and Legal Systems: A Short Survey](https://mlsanity.com/paper/2404.00990/) (2024)
- [TrustAI at SemEval-2024 Task 8: A Comprehensive Analysis of Multi-domain Machine Generated Text Detection Techniques](https://mlsanity.com/paper/2403.16592/) (2024)
- [Can large language models understand uncommon meanings of common words?](https://mlsanity.com/paper/2405.05741/) (2024)
- [How LLMs Comprehend Temporal Meaning in Narratives: A Case Study in Cognitive Evaluation of LLMs](https://mlsanity.com/paper/2507.14307/) (2025)
- [De-amplifying Bias from Differential Privacy in Language Model Fine-tuning](https://mlsanity.com/paper/2402.04489/) (2024)
- [When Not to Answer: Evaluating Prompts on GPT Models for Effective Abstention in Unanswerable Math Word Problems](https://mlsanity.com/paper/2410.13029/) (2024)
- [The risks of risk-based AI regulation: taking liability seriously](https://mlsanity.com/paper/2311.14684/) (2023)
- [Beyond Accuracy: Rethinking Hallucination and Regulatory Response in Generative AI](https://mlsanity.com/paper/2509.13345/) (2025)
- [Logic-Parametric Neuro-Symbolic NLI: Controlling Logical Formalisms for Verifiable LLM Reasoning](https://mlsanity.com/paper/2601.05705/) (2026)

## Abstract (arXiv)

With the launch of ChatGPT, Large Language Models (LLMs) are shaking up our whole society, rapidly altering the way we think, create and live. For instance, the GPT integration in Bing has altered our approach to online searching. While nascent LLMs have many advantages, new legal and ethical risks are also emerging, stemming in particular from stochastic parrots and hallucination. The EU is the first and foremost jurisdiction that has focused on the regulation of AI models. However, the risks posed by the new LLMs are likely to be underestimated by the emerging EU regulatory paradigm. Therefore, this correspondence warns that the European AI regulatory paradigm must evolve further to mitigate such risks.