mlsanityData to 14 Oct 2026

SsciBERT: A Pre-trained Language Model for Social Science Texts

arXiv 2206.04510 ↗First posted 9 Jun 2022cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

The academic literature of social sciences records human civilization and studies human social problems. With its large-scale growth, the ways to quickly find existing research on relevant issues have become an urgent demand for researchers. Previous studies, such as SciBERT, have shown that pre-training using domain-specific texts can improve the performance of natural language processing tasks. However, the pre-trained language model for social sciences is not available so far. In light of this, the present research proposes a pre-trained model based on the abstracts published in the Social Science Citation Index (SSCI) journals. The models, which are available on GitHub (https://github.com/S-T-Full-Text-Knowledge-Mining/SSCI-BERT), show excellent performance on discipline classification, abstract structure-function recognition, and named entity recognition tasks with the social sciences literature.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 29 of its 47 references that are in the map. First posted on arXiv 9 Jun 2022.

Builds on

29 of 47 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  6. 2018Deep contextualized word representationscited by 2,290
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  8. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  9. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  10. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  11. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 373
  12. 2020Explainable Automated Fact-Checking for Public Health Claimscited by 211
17 more
  1. 2020FinBERT: A Pretrained Language Model for Financial Communicationscited by 161
  2. 2020LEGAL-BERT: The Muppets straight out of Law Schoolcited by 111
  3. 2020GREEK-BERT: The Greeks visiting Sesame Streetcited by 38
  4. 2020The SOFC-Exp Corpus and Neural Approaches to Information Extraction in the Materials Science Domaincited by 32
  5. 2021MultiCite: Modeling realistic citations requires moving beyond the single-sentence single-label settingcited by 30
  6. 2020ExpBERT: Representation Engineering with Natural Language Explanationscited by 28
  7. 2021ParaSCI: A Large Scientific Paraphrase Dataset for Longer Paraphrase Generationcited by 28
  8. 2021CitationIE: Leveraging the Citation Graph for Scientific Information Extractioncited by 19
  9. 2021CiteWorth: Cite-Worthiness Detection for Improved Scientific Document Understandingcited by 17
  10. 2021SemEval-2021 Task 11: NLPContributionGraph -- Structuring Scholarly NLP Contributions for a Research Knowledge Graphcited by 16
  11. 2020Annotating and Extracting Synthesis Process of All-Solid-State Batteries from Scientific Literaturecited by 15
  12. 2021SciCo: Hierarchical Cross-Document Coreference for Scientific Conceptscited by 14
  13. 2020Natural Language Premise Selection: Finding Supporting Statements for Mathematical Textcited by 13
  14. 2020Domain-independent Extraction of Scientific Concepts from Research Articlescited by 11
  15. 2020The STEM-ECR Dataset: Grounding Scientific Entity References in STEM Scholarly Content to Authoritative Encyclopedic and Lexicographic Sourcescited by 9
  16. 2020SChuBERT: Scholarly Document Chunks with BERT-encoding boost Citation Count Predictioncited by 9
  17. 2020Biomedical Event Extraction with Hierarchical Knowledge Graphscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 4; 2025: 3; 2026: 1.

12 papers.

  1. 2023An Interdisciplinary Outlook on Large Language Models for Scientific Researchcited by 15
  2. 2024Scalable Training of Trustworthy and Energy-Efficient Predictive Graph Foundation Models for Atomistic Materials Modeling: A Case Study with HydraGNNcited by 8
  3. 2023When Large Language Models Meet Citation: A Surveycited by 6
  4. 2022Overview of the SV-Ident 2022 Shared Task on Survey Variable Identification in Social Science Publicationscited by 3
  5. 2025Advances in Pre-trained Language Models for Domain-Specific Text Classification: A Systematic Reviewcited by 3
  6. 2025Annotating Scientific Uncertainty: A comprehensive model using linguistic patterns and comparison with existing approachescited by 2
  7. 2023Embedding Models for Supervised Automatic Extraction and Classification of Named Entities in Scientific Acknowledgementscited by 0
  8. 2024Classifying populist language in American presidential and governor speeches using automatic text analysiscited by 0
  9. 2024Towards understanding evolution of science through language model seriescited by 0
  10. 2024Enriching Social Science Research via Survey Item Linkingcited by 0
  11. 2025Semantic Embeddings of Chemical Elements for Enhanced Materials Inference and Discoverycited by 0
  12. 2026SHARE: Social-Humanities AI for Research and Educationcited by 0