# Exploring the Potential of the Large Language Models (LLMs) in Identifying Misleading News Headlines

arXiv: https://arxiv.org/abs/2405.03153 · HTML: https://mlsanity.com/paper/2405.03153/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 17 references that are in the map.
- First posted on arXiv 6 May 2024.

## Builds on (most cited first, 3 of 3)

- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy](https://mlsanity.com/paper/2002.04087/) (2020)
- [Detecting Incongruity Between News Headline and Body Text via a Deep Hierarchical Encoder](https://mlsanity.com/paper/1811.07066/) (2018)

## Built on by (most cited first, 1 of 1)

- [Large Language Models' Detection of Political Orientation in Newspapers](https://mlsanity.com/paper/2406.00018/) (2024)

## Abstract (arXiv)

In the digital age, the prevalence of misleading news headlines poses a significant challenge to information integrity, necessitating robust detection mechanisms. This study explores the efficacy of Large Language Models (LLMs) in identifying misleading versus non-misleading news headlines. Utilizing a dataset of 60 articles, sourced from both reputable and questionable outlets across health, science & tech, and business domains, we employ three LLMs- ChatGPT-3.5, ChatGPT-4, and Gemini-for classification. Our analysis reveals significant variance in model performance, with ChatGPT-4 demonstrating superior accuracy, especially in cases with unanimous annotator agreement on misleading headlines. The study emphasizes the importance of human-centered evaluation in developing LLMs that can navigate the complexities of misinformation detection, aligning technical proficiency with nuanced human judgment. Our findings contribute to the discourse on AI ethics, emphasizing the need for models that are not only technically advanced but also ethically aligned and sensitive to the subtleties of human interpretation.