mlsanityData to 7 Oct 2026

L3Cube-IndicNews: News-based Short Text and Long Document Classification Datasets in Indic Languages

arXiv 2401.02254 ↗First posted 4 Jan 2024cs.CL cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

In this work, we introduce L3Cube-IndicNews, a multilingual text classification corpus aimed at curating a high-quality dataset for Indian regional languages, with a specific focus on news headlines and articles. We have centered our work on 10 prominent Indic languages, including Hindi, Bengali, Marathi, Telugu, Tamil, Gujarati, Kannada, Odia, Malayalam, and Punjabi. Each of these news datasets comprises 10 or more classes of news articles. L3Cube-IndicNews offers 3 distinct datasets tailored to handle different document lengths that are classified as: Short Headlines Classification (SHC) dataset containing the news headline and news category, Long Document Classification (LDC) dataset containing the whole news article and the news category, and Long Paragraph Classification (LPC) containing sub-articles of the news and the news category. We maintain consistent labeling across all 3 datasets for in-depth length-based analysis. We evaluate each of these Indic language datasets using 4 different models including monolingual BERT, multilingual Indic Sentence BERT (IndicSBERT), and IndicBERT. This research contributes significantly to expanding the pool of available text classification datasets and also makes it possible to develop topic classification models for Indian regional languages. This also serves as an excellent resource for cross-lingual analysis owing to the high overlap of labels among languages. The datasets and models are shared publicly at https://github.com/l3cube-pune/indic-nlp

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 12 of its 18 references that are in the map. First posted on arXiv 4 Jan 2024.

Builds on

12 of 18 references.

References · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2020Longformer: The Long-Document Transformercited by 3,362
  5. 2021MuRIL: Multilingual Representations for Indian Languagescited by 177
  6. 2022L3Cube-MahaCorpus and MahaBERT: Marathi Monolingual Corpus, Marathi BERT Language Models, and Resourcescited by 49
  7. 2020AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languagescited by 43
  8. 2022L3Cube-HindBERT and DevBERT: Pre-Trained BERT Transformer models for Devanagari based Hindi and Marathi Languagescited by 35
  9. 2020iNLTK: Natural Language Toolkit for Indic Languagescited by 24
  10. 2021Experimental Evaluation of Deep Learning models for Marathi Text Classificationcited by 19
  11. 2023L3Cube-IndicSBERT: A simple approach for learning cross-lingual sentence representations using multilingual BERTcited by 15
  12. 2022L3Cube-MahaSBERT and HindSBERT: Sentence BERT Models and Benchmarking BERT Sentence Representations for Hindi and Marathicited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 1.

8 papers.

  1. 2024On Limitations of LLM as Annotator for Low Resource Languagescited by 12
  2. 2025Bhaasha, Bhasa, Zaban: A Survey for Low-Resourced Languages in South Asia -- Current Stage and Challengescited by 6
  3. 2024Chain-of-Translation Prompting (CoTR): A Novel Prompting Technique for Low Resource Languagescited by 5
  4. 2024On Importance of Pruning and Distillation for Efficient Low Resource NLPcited by 1
  5. 2025Improving the Efficiency of Long Document Classification using Sentence Ranking Approachcited by 1
  6. 2025Efficient Zero-Shot Long Document Classification by Reducing Context Through Sentence Rankingcited by 0
  7. 2025L3Cube-IndicHeadline-ID: A Dataset for Headline Identification and Semantic Evaluation in Low-Resource Indian Languagescited by 0
  8. 2026MOSLD-Bench: Multilingual Open-Set Learning and Discovery Benchmark for Text Categorizationcited by 0