mlsanityData to 7 Oct 2026

LLM-Generated Negative News Headlines Dataset: Creation and Benchmarking Against Real Journalism

arXiv 2511.11591 ↗First posted 24 Oct 2025cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

This research examines the potential of datasets generated by Large Language Models (LLMs) to support Natural Language Processing (NLP) tasks, aiming to overcome challenges related to data acquisition and privacy concerns associated with real-world data. Focusing on negative valence text, a critical component of sentiment analysis, we explore the use of LLM-generated synthetic news headlines as an alternative to real-world data. A specialized corpus of negative news headlines was created using tailored prompts to capture diverse negative sentiments across various societal domains. The synthetic headlines were validated by expert review and further analyzed in embedding space to assess their alignment with real-world negative news in terms of content, tone, length, and style. Key metrics such as correlation with real headlines, perplexity, coherence, and realism were evaluated. The synthetic dataset was benchmarked against two sets of real news headlines using evaluations including the Comparative Perplexity Test, Comparative Readability Test, Comparative POS Profiling, BERTScore, and Comparative Semantic Similarity. Results show the generated headlines match real headlines with the only marked divergence being in the proper noun score of the POS profile test.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 34 references that are in the map. First posted on arXiv 24 Oct 2025.

Builds on

20 of 34 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  7. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  8. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  9. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 240
  10. 2023Does Synthetic Data Generation of LLMs Help Clinical Text Mining?cited by 151
  11. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitationscited by 96
  12. 2020Evolution of Semantic Similarity -- A Surveycited by 83
8 more
  1. 2023Contrasting Linguistic Patterns in Human and LLM-Generated News Textcited by 62
  2. 2023Neural Retrievers are Biased Towards LLM-Generated Contentcited by 37
  3. 2024Under the Surface: Tracking the Artifactuality of LLM-Generated Datacited by 23
  4. 2023Fake News Detectors are Biased against Texts Generated by Large Language Modelscited by 18
  5. 2024Enhancing Trust in LLMs: Algorithms for Comparing and Interpreting LLMscited by 9
  6. 2022MN-DS: A Multilabeled News Dataset for News Articles Hierarchical Classificationcited by 8
  7. 2019A new evaluation framework for topic modeling algorithms based on synthetic corporacited by 3
  8. 2023Grading Conversational Responses Of Chatbotscited by 1

Built on by

Not cited yet.