# Prompting or Fine-tuning? A Comparative Study of Large Language Models for Taxonomy Construction

arXiv: https://arxiv.org/abs/2309.01715 · HTML: https://mlsanity.com/paper/2309.01715/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 4 Sep 2023.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning](https://mlsanity.com/paper/2012.13255/) (2020)
- [Measuring the Intrinsic Dimension of Objective Landscapes](https://mlsanity.com/paper/1804.08838/) (2018)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [Constructing Taxonomies from Pretrained Language Models](https://mlsanity.com/paper/2010.12813/) (2020)
- [End-to-End Reinforcement Learning for Automatic Taxonomy Induction](https://mlsanity.com/paper/1805.04044/) (2018)
- [Towards using Few-Shot Prompt Learning for Automating Model Completion](https://mlsanity.com/paper/2212.03404/) (2022)
- [Distilling Hypernymy Relations from Language Models: On the Effectiveness of Zero-Shot Taxonomy Induction](https://mlsanity.com/paper/2202.04876/) (2022)
- [Recommending Metamodel Concepts during Modeling Activities with Pre-Trained Language Models](https://mlsanity.com/paper/2104.01642/) (2021)

## Built on by (most cited first, 18 of 18)

- [Chain-of-Layer: Iteratively Prompting Large Language Models for Taxonomy Induction from Limited Examples](https://mlsanity.com/paper/2402.07386/) (2024)
- [TaxoAdapt: Aligning LLM-Based Multidimensional Taxonomy Construction to Evolving Research Corpora](https://mlsanity.com/paper/2506.10737/) (2025)
- [From Prompt Engineering to Prompt Science With Human in the Loop](https://mlsanity.com/paper/2401.04122/) (2024)
- [From Papers to Panoramas: Building Hierarchies of Scientific Literature at Scale](https://mlsanity.com/paper/2504.13834/) (2025)
- [Advancing Dialectal Arabic to Modern Standard Arabic Machine Translation](https://mlsanity.com/paper/2507.20301/) (2025)
- [Refining Wikidata Taxonomy using Large Language Models](https://mlsanity.com/paper/2409.04056/) (2024)
- [LLMTaxo: Leveraging Large Language Models for Constructing Taxonomy of Factual Claims from Social Media](https://mlsanity.com/paper/2504.12325/) (2025)
- [Beyond True or False: Retrieval-Augmented Hierarchical Analysis of Nuanced Claims](https://mlsanity.com/paper/2506.10728/) (2025)
- [Accurate and Consistent Graph Model Generation from Text with Large Language Models](https://mlsanity.com/paper/2508.00255/) (2025)
- [Taxonomy Inference for Tabular Data Using Large Language Models](https://mlsanity.com/paper/2503.21810/) (2025)
- [On Active Privacy Auditing in Supervised Fine-tuning for White-Box Language Models](https://mlsanity.com/paper/2411.07070/) (2024)
- [CHisAgent: A Multi-Agent Framework for Event Taxonomy Construction in Ancient Chinese Cultural Systems](https://mlsanity.com/paper/2601.05520/) (2026)
- [How to Ask the AI: A User Perspective Survey for Large Language Model Prompting](https://mlsanity.com/paper/2608.07494/) (2026)
- [Jal Anveshak: Prediction of fishing zones using fine-tuned LlaMa 2](https://mlsanity.com/paper/2411.10050/) (2024)
- [FoodTaxo: Generating Food Taxonomies with Large Language Models](https://mlsanity.com/paper/2505.19838/) (2025)
- [Suffix-Constrained Greedy Search Algorithms for Causal Language Models](https://mlsanity.com/paper/2603.01243/) (2026)
- [BoostTaxo: Zero-Shot Taxonomy Induction via Boosting-Style Agentic Reasoning and Constraint-Aware Calibration](https://mlsanity.com/paper/2605.12520/) (2026)
- [SPARROW: Scalable Taxonomy Induction via Structure-Preserving Partitioning and Constraint-Guided Merging](https://mlsanity.com/paper/2609.07307/) (2026)

## Abstract (arXiv)

Taxonomies represent hierarchical relations between entities, frequently applied in various software modeling and natural language processing (NLP) activities. They are typically subject to a set of structural constraints restricting their content. However, manual taxonomy construction can be time-consuming, incomplete, and costly to maintain. Recent studies of large language models (LLMs) have demonstrated that appropriate user inputs (called prompting) can effectively guide LLMs, such as GPT-3, in diverse NLP tasks without explicit (re-)training. However, existing approaches for automated taxonomy construction typically involve fine-tuning a language model by adjusting model parameters. In this paper, we present a general framework for taxonomy construction that takes into account structural constraints. We subsequently conduct a systematic comparison between the prompting and fine-tuning approaches performed on a hypernym taxonomy and a novel computer science taxonomy dataset. Our result reveals the following: (1) Even without explicit training on the dataset, the prompting approach outperforms fine-tuning-based approaches. Moreover, the performance gap between prompting and fine-tuning widens when the training dataset is small. However, (2) taxonomies generated by the fine-tuning approach can be easily post-processed to satisfy all the constraints, whereas handling violations of the taxonomies produced by the prompting approach can be challenging. These evaluation findings provide guidance on selecting the appropriate method for taxonomy construction and highlight potential enhancements for both approaches.