# Multi-Layer Ranking with Large Language Models for News Source Recommendation

arXiv: https://arxiv.org/abs/2406.11745 · HTML: https://mlsanity.com/paper/2406.11745/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 33 references that are in the map.
- First posted on arXiv 17 Jun 2024.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence](https://mlsanity.com/paper/2103.08541/) (2021)
- [Explainable Automated Fact-Checking for Public Health Claims](https://mlsanity.com/paper/2010.09926/) (2020)
- [Is ChatGPT a Good Recommender? A Preliminary Study](https://mlsanity.com/paper/2304.10149/) (2023)
- [MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims](https://mlsanity.com/paper/1909.03242/) (2019)
- [Uncovering ChatGPT's Capabilities in Recommender Systems](https://mlsanity.com/paper/2305.02182/) (2023)
- [Simple BERT Models for Relation Extraction and Semantic Role Labeling](https://mlsanity.com/paper/1904.05255/) (2019)
- [The Fact Extraction and VERification (FEVER) Shared Task](https://mlsanity.com/paper/1811.10971/) (2018)
- [COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic](https://mlsanity.com/paper/2106.03794/) (2021)
- [Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programs](https://mlsanity.com/paper/2306.03081/) (2023)
- [Generative Recommendation: Towards Next-generation Recommender Paradigm](https://mlsanity.com/paper/2304.03516/) (2023)
- [Scientific Fact-Checking: A Survey of Resources and Approaches](https://mlsanity.com/paper/2305.16859/) (2023)
- [The Role of User Profile for Fake News Detection](https://mlsanity.com/paper/1904.13355/) (2019)
- [Fully Automated Fact Checking Using External Sources](https://mlsanity.com/paper/1710.00341/) (2017)
- [DirectQuote: A Dataset for Direct Quotation Extraction and Attribution in News Articles](https://mlsanity.com/paper/2110.07827/) (2021)
- [Some Observations on Fact-Checking Work with Implications for Computational Support](https://mlsanity.com/paper/2305.02224/) (2023)
- [Quote Erat Demonstrandum: A Web Interface for Exploring the Quotebank Corpus](https://mlsanity.com/paper/2207.03592/) (2022)
- [Neural Duplicate Question Detection without Labeled Training Data](https://mlsanity.com/paper/1911.05594/) (2019)
- [DIANES: A DEI Audit Toolkit for News Sources](https://mlsanity.com/paper/2203.11383/) (2022)

## Built on by (most cited first, 2 of 2)

- [LlamaLens: Specialized Multilingual LLM for Analyzing News and Social Media Content](https://mlsanity.com/paper/2410.15308/) (2024)
- [Mapping Trustworthiness in Large Language Models: A Bibliometric Analysis Bridging Theory to Practice](https://mlsanity.com/paper/2503.04785/) (2025)

## Abstract (arXiv)

To seek reliable information sources for news events, we introduce a novel task of expert recommendation, which aims to identify trustworthy sources based on their previously quoted statements. To achieve this, we built a novel dataset, called NewsQuote, consisting of 23,571 quote-speaker pairs sourced from a collection of news articles. We formulate the recommendation task as the retrieval of experts based on their likelihood of being associated with a given query. We also propose a multi-layer ranking framework employing Large Language Models to improve the recommendation performance. Our results show that employing an in-context learning based LLM ranker and a multi-layer ranking-based filter significantly improve both the predictive quality and behavioural quality of the recommender system.