mlsanityData to 14 Oct 2026

FrenchNews-7: Benchmarking Cross-Publisher French News Editorial Desk Classification

arXiv 2608.18097 ↗First posted 8 Jun 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We present FrenchNews-7, a cross-publisher France-based French-language news editorial desk classification benchmark combining a large multi-outlet corpus, a URL-derived seven-class taxonomy, and a fine-tuned CamemBERT classifier. Labels are assigned via a hybrid pipeline combining publisher URL slugs with LLM annotation for structurally ambiguous cases, audited through an inter-rater study (2 humans + 2 LLMs; pairwise kappa ≥ 0.766, human--human kappa = 0.806). We evaluate lexical, multilingual, and French-specific trained classifiers under both in-distribution and held-out-publisher settings, with additional comparison against zero-shot LLM baselines (GPT-OSS-120B, Mistral Small 3.2, Llama-3.3-70B) on the held-out pool. The strongest model, CamemBERT-base on full article text, outperforms headline-only input, generalizes to unseen outlets, and exceeds all three zero-shot LLM baselines on overall recall (0.799), with the gap concentrated in the ambiguous editorial-boundary categories Economie and Societe. Cross-publisher evaluation reveals uneven boundary stability: Sport, Culture & Loisirs, and International transfer cleanly, while Economie (recall = 0.517) is close to blinded human agreement (0.55), and Societe (precision = 0.577) absorbs boundary ambiguity, both suggesting editorial conventions rather than recoverable classifier headroom. The fine-tuned CamemBERT-base model, labeled manifest, reference collection scripts, and a reliability-tier guidance table are available at https://huggingface.co/LeFrenchNewsLab/camembert-base-frenchnews7 (model) and https://huggingface.co/datasets/LeFrenchNewsLab/frenchnews-7 (dataset).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 23 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

16 of 23 references.

References · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  4. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  5. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  6. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  7. 2019CamemBERT: a Tasty French Language Modelcited by 368
  8. 2022Is GPT-3 a Good Data Annotator?cited by 208
  9. 2019FlauBERT: Unsupervised Language Model Pre-training for Frenchcited by 153
  10. 2020MLSUM: The Multilingual Summarization Corpuscited by 121
  11. 2018Framing and Agenda-setting in Russian News: a Computational Analysis of Intricate Political Strategiescited by 47
  12. 2024Knowledge Distillation in Automated Annotation: Supervised Text Classification with LLM-Generated Training Labelscited by 25
4 more
  1. 2024CamemBERT 2.0: A Smarter French Language Model Aged to Perfectioncited by 14
  2. 2024LLM Teacher-Student Framework for Text Classification With No Manually Annotated Data: A Case Study in IPTC News Topic Classificationcited by 6
  3. 2024Automatic Classification of News Subjects in Broadcast News: Application to a Gender Bias Representation Analysiscited by 2
  4. 2025Feeding LLM Annotations to BERT Classifiers at Your Own Riskcited by 2

Built on by

Not cited yet.