# Knowledge Bases in Support of Large Language Models for Processing Web News

arXiv: https://arxiv.org/abs/2411.08278 · HTML: https://mlsanity.com/paper/2411.08278/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 56 references that are in the map.
- First posted on arXiv 13 Nov 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [How to Fine-Tune BERT for Text Classification?](https://mlsanity.com/paper/1905.05583/) (2019)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [K-BERT: Enabling Language Representation with Knowledge Graph](https://mlsanity.com/paper/1909.07606/) (2019)
- [BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1904.02232/) (2019)
- [WebSRC: A Dataset for Web-Based Structural Reading Comprehension](https://mlsanity.com/paper/2101.09465/) (2021)
- [Neural Open Information Extraction](https://mlsanity.com/paper/1805.04270/) (2018)
- [N24News: A New Dataset for Multimodal News Classification](https://mlsanity.com/paper/2108.13327/) (2021)
- [WebFormer: The Web-page Transformer for Structure Information Extraction](https://mlsanity.com/paper/2202.00217/) (2022)
- [TCN: Table Convolutional Network for Web Table Interpretation](https://mlsanity.com/paper/2102.09460/) (2021)
- [FreeDOM: A Transferable Neural Architecture for Structured Information Extraction on Web Documents](https://mlsanity.com/paper/2010.10755/) (2020)
- [Knowledge Graph informed Fake News Classification via Heterogeneous Representation Ensembles](https://mlsanity.com/paper/2110.10457/) (2021)
- [Getting More Out Of Syntax with PropS](https://mlsanity.com/paper/1603.01648/) (2016)
- [Relphormer: Relational Graph Transformer for Knowledge Graph Representations](https://mlsanity.com/paper/2205.10852/) (2022)

## Abstract (arXiv)

Large Language Models (LLMs) have received considerable interest in wide applications lately. During pre-training via massive datasets, such a model implicitly memorizes the factual knowledge of trained datasets in its hidden parameters. However, knowledge held implicitly in parameters often makes its use by downstream applications ineffective due to the lack of common-sense reasoning. In this article, we introduce a general framework that permits to build knowledge bases with an aid of LLMs, tailored for processing Web news. The framework applies a rule-based News Information Extractor (NewsIE) to news items for extracting their relational tuples, referred to as knowledge bases, which are then graph-convoluted with the implicit knowledge facts of news items obtained by LLMs, for their classification. It involves two lightweight components: 1) NewsIE: for extracting the structural information of every news item, in the form of relational tuples; 2) BERTGraph: for graph convoluting the implicit knowledge facts with relational tuples extracted by NewsIE. We have evaluated our framework under different news-related datasets for news category classification, with promising experimental results.