mlsanityData to 14 Oct 2026

Knowledge Bases in Support of Large Language Models for Processing Web News

arXiv 2411.08278 ↗First posted 13 Nov 2024cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have received considerable interest in wide applications lately. During pre-training via massive datasets, such a model implicitly memorizes the factual knowledge of trained datasets in its hidden parameters. However, knowledge held implicitly in parameters often makes its use by downstream applications ineffective due to the lack of common-sense reasoning. In this article, we introduce a general framework that permits to build knowledge bases with an aid of LLMs, tailored for processing Web news. The framework applies a rule-based News Information Extractor (NewsIE) to news items for extracting their relational tuples, referred to as knowledge bases, which are then graph-convoluted with the implicit knowledge facts of news items obtained by LLMs, for their classification. It involves two lightweight components: 1) NewsIE: for extracting the structural information of every news item, in the form of relational tuples; 2) BERTGraph: for graph convoluting the implicit knowledge facts with relational tuples extracted by NewsIE. We have evaluated our framework under different news-related datasets for news category classification, with promising experimental results.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 56 references that are in the map. First posted on arXiv 13 Nov 2024.

Builds on

23 of 56 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  7. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
  8. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,102
  9. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 723
  10. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 546
  11. 2019How to Fine-Tune BERT for Text Classification?cited by 411
  12. 2016End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRFcited by 341
11 more
  1. 2019K-BERT: Enabling Language Representation with Knowledge Graphcited by 304
  2. 2019BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysiscited by 176
  3. 2021WebSRC: A Dataset for Web-Based Structural Reading Comprehensioncited by 100
  4. 2018Neural Open Information Extractioncited by 50
  5. 2021N24News: A New Dataset for Multimodal News Classificationcited by 40
  6. 2022WebFormer: The Web-page Transformer for Structure Information Extractioncited by 34
  7. 2021TCN: Table Convolutional Network for Web Table Interpretationcited by 24
  8. 2020FreeDOM: A Transferable Neural Architecture for Structured Information Extraction on Web Documentscited by 22
  9. 2021Knowledge Graph informed Fake News Classification via Heterogeneous Representation Ensemblescited by 14
  10. 2016Getting More Out Of Syntax with PropScited by 8
  11. 2022Relphormer: Relational Graph Transformer for Knowledge Graph Representationscited by 7

Built on by

Not cited yet.