# Misinforming LLMs: vulnerabilities, challenges and opportunities

arXiv: https://arxiv.org/abs/2408.01168 · HTML: https://mlsanity.com/paper/2408.01168/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 12 references that are in the map.
- First posted on arXiv 2 Aug 2024.

## Builds on (most cited first, 5 of 5)

- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning](https://mlsanity.com/paper/2402.06332/) (2024)
- [Uncovering Logit Suppression Vulnerabilities in LLM Safety Alignment](https://mlsanity.com/paper/2405.13068/) (2024)
- [Arithmetic Reasoning with LLM: Prolog Generation & Permutation](https://mlsanity.com/paper/2405.17893/) (2024)

## Built on by (most cited first, 4 of 4)

- [Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attacks](https://mlsanity.com/paper/2409.08087/) (2024)
- [Recursive Decomposition of Logical Thoughts: Framework for Superior Reasoning and Knowledge Propagation in Large Language Models](https://mlsanity.com/paper/2501.02026/) (2025)
- [Enhancing Scientific Reproducibility Through Automated BioCompute Object Creation Using Retrieval-Augmented Generation from Publications](https://mlsanity.com/paper/2409.15076/) (2024)
- [Curation of a Cardiology Interface Terminology for Highlighting Electronic Health Records using Machine Learning](https://mlsanity.com/paper/2606.08311/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have made significant advances in natural language processing, but their underlying mechanisms are often misunderstood. Despite exhibiting coherent answers and apparent reasoning behaviors, LLMs rely on statistical patterns in word embeddings rather than true cognitive processes. This leads to vulnerabilities such as "hallucination" and misinformation. The paper argues that current LLM architectures are inherently untrustworthy due to their reliance on correlations of sequential patterns of word embedding vectors. However, ongoing research into combining generative transformer-based models with fact bases and logic programming languages may lead to the development of trustworthy LLMs capable of generating statements based on given truth and explaining their self-reasoning process.