mlsanityData to 7 Oct 2026

L3Cube-MahaCorpus and MahaBERT: Marathi Monolingual Corpus, Marathi BERT Language Models, and Resources

arXiv 2202.01159 ↗First posted 2 Feb 2022cs.CL cs.LGcited by 49

Not on the map. Why?

Abstract (arXiv)

We present L3Cube-MahaCorpus a Marathi monolingual data set scraped from different internet sources. We expand the existing Marathi monolingual corpus with 24.8M sentences and 289M tokens. We further present, MahaBERT, MahaAlBERT, and MahaRoBerta all BERT-based masked language models, and MahaFT, the fast text word embeddings both trained on full Marathi corpus with 752M tokens. We show the effectiveness of these resources on downstream Marathi sentiment analysis, text classification, and named entity recognition (NER) tasks. We also release MahaGPT, a generative Marathi GPT model trained on Marathi corpus. Marathi is a popular language in India but still lacks these resources. This work is a step forward in building open resources for the Marathi language. The data and models are available at https://github.com/l3cube-pune/MarathiNLP .

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 24 of its 34 references that are in the map. First posted on arXiv 2 Feb 2022.

Builds on

24 of 34 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  5. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  6. 2019How multilingual is Multilingual BERT?cited by 642
  7. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  8. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  9. 2020ARBERT & MARBERT: Deep Bidirectional Transformers for Arabiccited by 164
  10. 2019FlauBERT: Unsupervised Language Model Pre-training for Frenchcited by 153
  11. 2020PhoBERT: Pre-trained language models for Vietnamesecited by 148
  12. 2020RobBERT: a Dutch RoBERTa-based Language Modelcited by 88
12 more
  1. 2020GottBERT: a pure German Language Modelcited by 49
  2. 2021RobeCzech: Czech RoBERTa, a monolingual contextualized language representation modelcited by 33
  3. 2021L3CubeMahaSent: A Marathi Tweet-based Sentiment Analysis Datasetcited by 28
  4. 2021Cross-lingual Offensive Language Identification for Low Resource Languages: The Case of Marathicited by 28
  5. 2020iNLTK: Natural Language Toolkit for Indic Languagescited by 24
  6. 2020Deep Learning for Hindi Text Classification: A Comparisoncited by 22
  7. 2020Indic-Transformers: An Analysis of Transformer Language Models for Indian Languagescited by 22
  8. 2021Experimental Evaluation of Deep Learning models for Marathi Text Classificationcited by 19
  9. 2021Hate and Offensive Speech Detection in Hindi and Marathicited by 17
  10. 2021Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languagescited by 17
  11. 2021Evaluation of Deep Learning Models for Hostility Detection in Hindi Textcited by 7
  12. 2021Contextual Hate Speech Detection in Code Mixed Text using Transformer Based Approachescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 11; 2023: 12; 2024: 12; 2025: 9; 2026: 5.

49 papers.

Sort:

  1. 2022L3Cube-HindBERT and DevBERT: Pre-Trained BERT Transformer models for Devanagari based Hindi and Marathi Languagescited by 35
  2. 2022L3Cube-MahaNLP: Marathi Natural Language Processing Datasets, Models, and Librarycited by 30
  3. 2022L3Cube-MahaNER: A Marathi Named Entity Recognition Dataset and BERT modelscited by 21
  4. 2022L3Cube-MahaHate: A Tweet-based Marathi Hate Speech Detection Dataset and BERT modelscited by 20
  5. 2022Mono vs Multilingual BERT for Hate Speech Detection and Text Classification: A Case Study in Marathicited by 16
  6. 2023L3Cube-IndicSBERT: A simple approach for learning cross-lingual sentence representations using multilingual BERTcited by 15
  7. 2022L3Cube-MahaSBERT and HindSBERT: Sentence BERT Models and Benchmarking BERT Sentence Representations for Hindi and Marathicited by 10
  8. 2024L3Cube-IndicNews: News-based Short Text and Long Document Classification Datasets in Indic Languagescited by 8
  9. 2023L3Cube-MahaSent-MD: A Multi-domain Marathi Sentiment Analysis Dataset and Transformer Modelscited by 6
  10. 2024L3Cube-MahaNews: News-based Short Text and Long Document Classification Datasets in Marathicited by 6
  11. 2025Bhaasha, Bhasa, Zaban: A Survey for Low-Resourced Languages in South Asia -- Current Stage and Challengescited by 6
  12. 2022Mono vs Multilingual BERT: A Case Study in Hindi and Marathi Named Entity Recognitioncited by 5
37 more
  1. 2022Overview of the HASOC Subtrack at FIRE 2022: Offensive Language Identification in Marathicited by 5
  2. 2024Chain-of-Translation Prompting (CoTR): A Novel Prompting Technique for Low Resource Languagescited by 5
  3. 2022Spread Love Not Hate: Undermining the Importance of Hateful Pre-training for Hate Speech Detectioncited by 4
  4. 2023Breaking Language Barriers: A Question Answering Dataset for Hindi and Marathicited by 4
  5. 2025IndicSQuAD: A Comprehensive Multilingual Question Answering Dataset for Indic Languagescited by 4
  6. 2022A Twitter BERT Approach for Offensive Language Detection in Marathicited by 3
  7. 2023Question answering using deep learning in low resource Indian language Marathicited by 3
  8. 2023mahaNLP: A Marathi Natural Language Processing Librarycited by 3
  9. 2023My Boli: Code-mixed Marathi-English Corpora, Pretrained Language Models and Evaluation Benchmarkscited by 2
  10. 2023L3Cube-MahaSocialNER: A Social Media based Marathi NER Dataset and BERT modelscited by 2
  11. 2024MasonTigers@LT-EDI-2024: An Ensemble Approach Towards Detecting Homophobia and Transphobia in Social Media Commentscited by 2
  12. 2022Towards Simple and Efficient Task-Adaptive Pre-training for Text Classificationcited by 1
  13. 2024MasonTigers at SemEval-2024 Task 1: An Ensemble Approach for Semantic Textual Relatednesscited by 1
  14. 2024MahaSQuAD: Bridging Linguistic Divides in Marathi Question-Answeringcited by 1
  15. 2024L3Cube-MahaSum: A Comprehensive Dataset and BART Models for Abstractive Text Summarization in Marathicited by 1
  16. 2024Long Range Named Entity Recognition for Marathi Documentscited by 1
  17. 2025Enhancing Plagiarism Detection in Marathi with a Weighted Ensemble of TF-IDF and BERT Embeddings for Low-Resource Language Processingcited by 1
  18. 2025L3Cube-MahaEmotions: A Marathi Emotion Recognition Dataset with Synthetic Annotations using CoTR prompting and Large Language Modelscited by 1
  19. 2025Improving the Efficiency of Long Document Classification using Sentence Ranking Approachcited by 1
  20. 2023Leveraging Language Identification to Enhance Code-Mixed Text Classificationcited by 0
  21. 2023Enhancing Low Resource NER Using Assisting Language And Transfer Learningcited by 0
  22. 2023Robust Sentiment Analysis for Low Resource languages Using Data Augmentation Approaches: A Case Study in Marathicited by 0
  23. 2023On Significance of Subword tokenization for Low Resource and Efficient Named Entity Recognition: A case study in Marathicited by 0
  24. 2023HCDIR: End-to-end Hate Context Detection, and Intensity Reduction model for online commentscited by 0
  25. 2024Curating Stopwords in Marathi: A TF-IDF Approach for Improved Text Analysis and Information Retrievalcited by 0
  26. 2024Leveraging Parameter Efficient Training Methods for Low Resource Text Classification: A Case Study in Marathicited by 0
  27. 2024Non-Contextual BERT or FastText? A Comparative Analysiscited by 0
  28. 2024A Review of the Marathi Natural Language Processingcited by 0
  29. 2025On Importance of Layer Pruning for Smaller BERT Models and Low Resource Languagescited by 0
  30. 2025MahaParaphrase: A Marathi Paraphrase Detection Corpus and BERT-based Modelscited by 0
  31. 2025Efficient Zero-Shot Long Document Classification by Reducing Context Through Sentence Rankingcited by 0
  32. 2025L3Cube-MahaSTS: A Marathi Sentence Similarity Dataset and Modelscited by 0
  33. 2026L3Cube-MahaPOS: A Marathi Part-of-Speech Tagging Dataset and BERT Modelscited by 0
  34. 2026Structure-Preserving Document Translation via Multi-Stage LLM Pipeline: A Case Study in Marathicited by 0
  35. 2026BERT-based Models vs. Large Language Models for Low-Resource Named Entity Recognition: A Comparative Study on Marathicited by 0
  36. 2026HomoEnsNER: Does Language Alignment Outperform Architectural Complexity in Gujarati Named Entity Recognition?cited by 0
  37. 2026Named Entity Recognition using Sliding Window Approachcited by 0