# SciPrompt: Knowledge-augmented Prompting for Fine-grained Categorization of Scientific Topics

arXiv: https://arxiv.org/abs/2410.01946 · HTML: https://mlsanity.com/paper/2410.01946/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Builds on (most cited first, 23 of 23)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [SPECTER: Document-level Representation Learning using Citation-informed Transformers](https://mlsanity.com/paper/2004.07180/) (2020)
- [True Few-Shot Learning with Language Models](https://mlsanity.com/paper/2105.11447/) (2021)
- [PTR: Prompt Tuning with Rules for Text Classification](https://mlsanity.com/paper/2105.11259/) (2021)
- [WARP: Word-level Adversarial ReProgramming](https://mlsanity.com/paper/2101.00121/) (2021)
- [Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification](https://mlsanity.com/paper/2108.02035/) (2021)
- [Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs](https://mlsanity.com/paper/2312.05934/) (2023)
- [Noisy Channel Language Model Prompting for Few-Shot Text Classification](https://mlsanity.com/paper/2108.04106/) (2021)
- [OpenPrompt: An Open-source Framework for Prompt-learning](https://mlsanity.com/paper/2111.01998/) (2021)
- [Prompt-Learning for Fine-Grained Entity Typing](https://mlsanity.com/paper/2108.10604/) (2021)
- [Weakly-Supervised Hierarchical Text Classification](https://mlsanity.com/paper/1812.11270/) (2018)
- [Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning](https://mlsanity.com/paper/2205.14704/) (2022)
- [AdaPrompt: Adaptive Model Training for Prompt-based NLP](https://mlsanity.com/paper/2202.04824/) (2022)
- [SciNLI: A Corpus for Natural Language Inference on Scientific Text](https://mlsanity.com/paper/2203.06728/) (2022)
- [Pre-trained Language Models Can be Fully Zero-Shot Learners](https://mlsanity.com/paper/2212.06950/) (2022)
- [Towards Unified Prompt Tuning for Few-shot Text Classification](https://mlsanity.com/paper/2205.05313/) (2022)
- [Beyond Binary Gender Labels: Revealing Gender Biases in LLMs through Gender-Neutral Name Predictions](https://mlsanity.com/paper/2407.05271/) (2024)
- [Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations](https://mlsanity.com/paper/2210.16637/) (2022)

## Built on by (most cited first, 4 of 4)

- [Improving Clinical Diagnosis with Counterfactual Multi-Agent Reasoning](https://mlsanity.com/paper/2603.27820/) (2026)
- [PlainQAFact: Retrieval-augmented Factual Consistency Evaluation Metric for Biomedical Plain Language Summarization](https://mlsanity.com/paper/2503.08890/) (2025)
- [Neuron-Level Interventions for Gendered and Gender-Neutral Generation in Language Models](https://mlsanity.com/paper/2605.30717/) (2026)
- [AutoTail-BSFGM: Class-Balance-Aware Fine-Tuning for Chinese Scholarly Text Classification](https://mlsanity.com/paper/2606.03576/) (2026)

## Abstract (arXiv)

Prompt-based fine-tuning has become an essential method for eliciting information encoded in pre-trained language models for a variety of tasks, including text classification. For multi-class classification tasks, prompt-based fine-tuning under low-resource scenarios has resulted in performance levels comparable to those of fully fine-tuning methods. Previous studies have used crafted prompt templates and verbalizers, mapping from the label terms space to the class space, to solve the classification problem as a masked language modeling task. However, cross-domain and fine-grained prompt-based fine-tuning with an automatically enriched verbalizer remains unexplored, mainly due to the difficulty and costs of manually selecting domain label terms for the verbalizer, which requires humans with domain expertise. To address this challenge, we introduce SciPrompt, a framework designed to automatically retrieve scientific topic-related terms for low-resource text classification tasks. To this end, we select semantically correlated and domain-specific label terms within the context of scientific literature for verbalizer augmentation. Furthermore, we propose a new verbalization strategy that uses correlation scores as additional weights to enhance the prediction performance of the language model during model tuning. Our method outperforms state-of-the-art, prompt-based fine-tuning methods on scientific text classification tasks under few and zero-shot settings, especially in classifying fine-grained and emerging scientific topics.