# The Generative AI Paradox: GenAI and the Erosion of Trust, the Corrosion of Information Verification, and the Demise of Truth

arXiv: https://arxiv.org/abs/2601.00306 · HTML: https://mlsanity.com/paper/2601.00306/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 38 references that are in the map.
- First posted on arXiv 1 Jan 2026.

## Builds on (most cited first, 14 of 14)

- [Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training](https://mlsanity.com/paper/2401.05566/) (2024)
- [Factuality Challenges in the Era of Large Language Models](https://mlsanity.com/paper/2310.05189/) (2023)
- [Fairness And Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, And Mitigation Strategies](https://mlsanity.com/paper/2304.07683/) (2023)
- [GenAI Against Humanity: Nefarious Applications of Generative Artificial Intelligence and Large Language Models](https://mlsanity.com/paper/2310.00737/) (2023)
- [Data Poisoning for In-context Learning](https://mlsanity.com/paper/2402.02160/) (2024)
- [Examining the Impact of Provenance-Enabled Media on Trust and Accuracy Perceptions](https://mlsanity.com/paper/2303.12118/) (2023)
- [Auditing Political Exposure Bias: Algorithmic Amplification on Twitter/X During the 2024 U.S. Presidential Election](https://mlsanity.com/paper/2411.01852/) (2024)
- [Exposing Cross-Platform Coordinated Inauthentic Activity in the Run-Up to the 2024 U.S. Election](https://mlsanity.com/paper/2410.22716/) (2024)
- [Synthetic Politics: Prevalence, Spreaders, and Emotional Reception of AI-Generated Political Images on X](https://mlsanity.com/paper/2502.11248/) (2025)
- [Disinformation 2.0 in the Age of AI: A Cybersecurity Perspective](https://mlsanity.com/paper/2306.05569/) (2023)
- [Community Moderation and the New Epistemology of Fact Checking on Social Media](https://mlsanity.com/paper/2505.20067/) (2025)
- [Coordinated Inauthentic Behavior on TikTok: Challenges and Opportunities for Detection in a Video-First Ecosystem](https://mlsanity.com/paper/2505.10867/) (2025)
- [Charting the Landscape of Nefarious Uses of Generative Artificial Intelligence for Online Election Interference](https://mlsanity.com/paper/2406.01862/) (2024)
- [Uncovering Coordinated Cross-Platform Information Operations Threatening the Integrity of the 2024 U.S. Presidential Election Online Discussion](https://mlsanity.com/paper/2409.15402/) (2024)

## Built on by (most cited first, 5 of 5)

- [Industrialized Deception: The Collateral Effects of LLM-Generated Misinformation on Digital Ecosystems](https://mlsanity.com/paper/2601.21963/) (2026)
- [The Impact of AI-Generated Text on the Internet](https://mlsanity.com/paper/2604.26965/) (2026)
- [MOSAIC: Unveiling the Moral, Social and Individual Dimensions of Large Language Models](https://mlsanity.com/paper/2603.00048/) (2026)
- [Mapping how LLMs debate societal issues when shadowing human personality traits, sociodemographics and social media behavior](https://mlsanity.com/paper/2604.27624/) (2026)
- [Studying Comics at Scale with Artificial Intelligence: The PAGE Framework and Three Studies of American Comics and Japanese Manga](https://mlsanity.com/paper/2610.13840/) (2026)

## Abstract (arXiv)

Generative AI (GenAI) now produces text, images, audio, and video that can be perceptually convincing at scale and at negligible marginal cost. While public debate often frames the associated harms as "deepfakes" or incremental extensions of misinformation and fraud, this view misses a broader socio-technical shift: GenAI enables synthetic realities; coherent, interactive, and potentially personalized information environments in which content, identity, and social interaction are jointly manufactured and mutually reinforcing. We argue that the most consequential risk is not merely the production of isolated synthetic artifacts, but the progressive erosion of shared epistemic ground and institutional verification practices as synthetic content, synthetic identity, and synthetic interaction become easy to generate and hard to audit. This paper (i) formalizes synthetic reality as a layered stack (content, identity, interaction, institutions), (ii) expands a taxonomy of GenAI harms spanning personal, economic, informational, and socio-technical risks, (iii) articulates the qualitative shifts introduced by GenAI (cost collapse, throughput, customization, micro-segmentation, provenance gaps, and trust erosion), and (iv) synthesizes recent risk realizations (2023-2025) into a compact case bank illustrating how these mechanisms manifest in fraud, elections, harassment, documentation, and supply-chain compromise. We then propose a mitigation stack that treats provenance infrastructure, platform governance, institutional workflow redesign, and public resilience as complementary rather than substitutable, and outline a research agenda focused on measuring epistemic security. We conclude with the Generative AI Paradox: as synthetic media becomes ubiquitous, societies may rationally discount digital evidence altogether.