K-12BERT: BERT for K-12 education
Not on the map. Why?
Abstract (arXiv)
Online education platforms are powered by various NLP pipelines, which utilize models like BERT to aid in content curation. Since the inception of the pre-trained language models like BERT, there have also been many efforts toward adapting these pre-trained models to specific domains. However, there has not been a model specifically adapted for the education domain (particularly K-12) across subjects to the best of our knowledge. In this work, we propose to train a language model on a corpus of data curated by us across multiple subjects from various sources for K-12 education. We also evaluate our model, K12-BERT, on downstream tasks like hierarchical taxonomy tagging.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 9 of its 9 references that are in the map. First posted on arXiv 24 May 2022.
Builds on
9 of 9 references.
References · 9
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
- 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
- 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
- 2018Universal Sentence Encodercited by 489
- 2019FinBERT: Financial Sentiment Analysis with Pre-trained Language Modelscited by 385
- 2019PatentBERT: Patent Classification with Fine-Tuning a pre-trained BERT Modelcited by 24
- 2021Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representationcited by 11
- 2021TagRec: Automated Tagging of Questions with Hierarchical Learning Taxonomycited by 6
Built on by
Citing papers per year: 2023: 1.
1 paper.