# The Tag is the Signal: URL-Agnostic Credibility Scoring for Messages on Telegram

arXiv: https://arxiv.org/abs/2601.13294 · HTML: https://mlsanity.com/paper/2601.13294/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 93 references that are in the map.
- First posted on arXiv 19 Jan 2026.

## Builds on (most cited first, 20 of 20)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [A Stylometric Inquiry into Hyperpartisan and Fake News](https://mlsanity.com/paper/1702.05638/) (2017)
- [The COVID-19 Social Media Infodemic](https://mlsanity.com/paper/2003.05004/) (2020)
- [This Just In: Fake News Packs a Lot in Title, Uses Simpler, Repetitive Content in Text Body, More Similar to Satire than Real News](https://mlsanity.com/paper/1703.09398/) (2017)
- [NELA-GT-2020: A Large Multi-Labelled News Dataset for The Study of Misinformation in News Articles](https://mlsanity.com/paper/2102.04567/) (2021)
- [Prta: A System to Support the Analysis of Propaganda Techniques in the News](https://mlsanity.com/paper/2005.05854/) (2020)
- [A Survey on Predicting the Factuality and the Bias of News Media](https://mlsanity.com/paper/2103.12506/) (2021)
- [Large Language Models for Propaganda Span Annotation](https://mlsanity.com/paper/2311.09812/) (2023)
- [The Anatomy of a Cryptocurrency Pump-and-Dump Scheme](https://mlsanity.com/paper/1811.10109/) (2018)
- [Weakly Supervised Veracity Classification with LLM-Predicted Credibility Signals](https://mlsanity.com/paper/2309.07601/) (2023)
- [The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotators](https://mlsanity.com/paper/2407.11004/) (2024)
- [Political Bias and Factualness in News Sharing across more than 100,000 Online Communities](https://mlsanity.com/paper/2102.08537/) (2021)
- [Alfred: A System for Prompted Weak Supervision](https://mlsanity.com/paper/2305.18623/) (2023)
- [DarkGram: A Large-Scale Analysis of Cybercriminal Activity Channels on Telegram](https://mlsanity.com/paper/2409.14596/) (2024)
- [GPT Assisted Annotation of Rhetorical and Linguistic Features for Interpretable Propaganda Technique Detection in News Text](https://mlsanity.com/paper/2407.11827/) (2024)
- [Characterizing and Detecting Propaganda-Spreading Accounts on Telegram](https://mlsanity.com/paper/2406.08084/) (2024)
- [Extracting Participation in Collective Action from Social Media](https://mlsanity.com/paper/2501.07368/) (2025)
- [Cybersecurity Misinformation Detection on Social Media: Case Studies on Phishing Reports and Zoom's Threats](https://mlsanity.com/paper/2110.12296/) (2021)

## Abstract (arXiv)

Telegram has become one of the leading platforms for disseminating misinformational messages. However, many existing pipelines still classify each message's credibility based on the reputation of its associated domain names or its lexical features. Such methods work well on traditional long-form news articles published by well-known sources, but high-risk posts on Telegram are short and URL-sparse, leading to failures for link-based and standard TF-IDF models. To this end, we propose the TAG2CRED pipeline, a method designed for such short, convoluted messages. Our model will directly score each post based on the tags assigned to the text. We designed a concise label system that covers the dimensions of theme, claim type, call to action, and evidence. The fine-tuned large language model (LLM) assigns tags to messages and then maps these tags to calibrated risk scores in the [0,1] interval through L2-regularized logistic regression. We evaluated 87,936 Telegram messages associated with Media Bias/Fact Check (MBFC), using URL masking and domain disjoint splits. The results showed that the ROC-AUC of the TAG2CRED model reached 0.871, the macro-F1 value was 0.787, and the Brier score was 0.167, outperforming the baseline TF-IDF (macro-F1 value 0.737, Brier score 0.248); at the same time, the number of features used in this model is much smaller, and the generalization ability on infrequent domains is stronger. The performance of the stacked ensemble model (TF-IDF + TAG2CRED + SBERT) was further improved over the baseline SBERT. ROC-AUC reached 0.901, and the macro-F1 value was 0.813 (Brier score 0.114). This indicates that style labels and lexical features may capture different but complementary dimensions of information risk.