# LLM Teacher-Student Framework for Text Classification With No Manually Annotated Data: A Case Study in IPTC News Topic Classification arXiv: https://arxiv.org/abs/2411.19638 ยท HTML: https://mlsanity.com/paper/2411.19638/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation. - Builds on 20 of its 57 references that are in the map. - First posted on arXiv 29 Nov 2024. ## Placed in - Knowledge distillation / [Knowledge distillation for LLMs](https://mlsanity.com/distill/knowledge-distillation-for-llms/) ## Builds on (most cited first, 20 of 20) - [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022) - [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019) - [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013) - [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022) - [Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision](https://mlsanity.com/paper/2305.03047/) (2023) - [Is GPT-3 a Good Data Annotator?](https://mlsanity.com/paper/2212.10450/) (2022) - [Large Language Models for Data Annotation and Synthesis: A Survey](https://mlsanity.com/paper/2402.13446/) (2024) - [Generating Training Data with Language Models: Towards Zero-Shot Language Understanding](https://mlsanity.com/paper/2202.04538/) (2022) - [Toxic Language Detection in Social Media for Brazilian Portuguese: New Dataset and Multilingual Analysis](https://mlsanity.com/paper/2010.04543/) (2020) - [News Category Dataset](https://mlsanity.com/paper/2209.11429/) (2022) - [Analyzing Dataset Annotation Quality Management in the Wild](https://mlsanity.com/paper/2307.08153/) (2023) - [CodecLM: Aligning Language Models with Tailored Synthetic Data](https://mlsanity.com/paper/2404.05875/) (2024) - [COPAL-ID: Indonesian Language Reasoning with Local Culture and Nuances](https://mlsanity.com/paper/2311.01012/) (2023) - [NELA-GT-2018: A Large Multi-Labelled News Dataset for The Study of Misinformation in News Articles](https://mlsanity.com/paper/1904.01546/) (2019) - [Media Cloud: Massive Open Source Collection of Global News on the Open Web](https://mlsanity.com/paper/2104.03702/) (2021) - [Learning Reward for Robot Skills Using Large Language Models via Self-Alignment](https://mlsanity.com/paper/2405.07162/) (2024) - [The FRENK Datasets of Socially Unacceptable Discourse in Slovene and English](https://mlsanity.com/paper/1906.02045/) (2019) - [MN-DS: A Multilabeled News Dataset for News Articles Hierarchical Classification](https://mlsanity.com/paper/2212.12061/) (2022) - [Evaluation of contextual embeddings on less-resourced languages](https://mlsanity.com/paper/2107.10614/) (2021) - [Do Language Models Care About Text Quality? Evaluating Web-Crawled Corpora Across 11 Languages](https://mlsanity.com/paper/2403.08693/) (2024) ## Built on by (most cited first, 6 of 6) - [State of the Art in Text Classification for South Slavic Languages: Fine-Tuning or Prompting?](https://mlsanity.com/paper/2511.07989/) (2025) - [Extending a Parliamentary Corpus with MPs' Tweets: Automatic Annotation and Evaluation Using MultiParTweet](https://mlsanity.com/paper/2512.11567/) (2025) - [Environmental, Social and Governance Sentiment Analysis on Slovene News: A Novel Dataset and Models](https://mlsanity.com/paper/2604.06826/) (2026) - [The Growing Gains and Pains of Iterative Web Corpora Crawling: Insights from South Slavic CLASSLA-web 2.0 Corpora](https://mlsanity.com/paper/2601.11170/) (2026) - [Supercharging Agenda Setting Research: The ParlaCAP Dataset of 28 European Parliaments and a Scalable Multilingual LLM-Based Classification](https://mlsanity.com/paper/2602.16516/) (2026) - [FrenchNews-7: Benchmarking Cross-Publisher French News Editorial Desk Classification](https://mlsanity.com/paper/2608.18097/) (2026) ## Abstract (arXiv) With the ever-increasing number of news stories available online, classifying them by topic, regardless of the language they are written in, has become crucial for enhancing readers' access to relevant content. To address this challenge, we propose a teacher-student framework based on large language models (LLMs) for developing multilingual news topic classification models of reasonable size with no need for manual data annotation. The framework employs a Generative Pretrained Transformer (GPT) model as the teacher model to develop a news topic training dataset through automatic annotation of 20,000 news articles in Slovenian, Croatian, Greek, and Catalan. Articles are classified into 17 main categories from the Media Topic schema, developed by the International Press Telecommunications Council (IPTC). The teacher model exhibits high zero-shot performance in all four languages. Its agreement with human annotators is comparable to that between the human annotators themselves. To mitigate the computational limitations associated with the requirement of processing millions of texts daily, smaller BERT-like student models are fine-tuned on the GPT-annotated dataset. These student models achieve high performance comparable to the teacher model. Furthermore, we explore the impact of the training data size on the performance of the student models and investigate their monolingual, multilingual, and zero-shot cross-lingual capabilities. The findings indicate that student models can achieve high performance with a relatively small number of training instances, and demonstrate strong zero-shot cross-lingual abilities. Finally, we publish the best-performing news topic classifier, enabling multilingual classification with the top-level categories of the IPTC Media Topic schema.