# Are Large Language Models a Good Replacement of Taxonomies?

arXiv: https://arxiv.org/abs/2406.11131 · HTML: https://mlsanity.com/paper/2406.11131/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 68 references that are in the map.
- First posted on arXiv 17 Jun 2024.

## Builds on (most cited first, 24 of 24)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models](https://mlsanity.com/paper/2305.08322/) (2023)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters](https://mlsanity.com/paper/2002.01808/) (2020)
- [Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?](https://mlsanity.com/paper/2308.10168/) (2023)
- [Simple Entity-Centric Questions Challenge Dense Retrievers](https://mlsanity.com/paper/2109.08535/) (2021)
- [Source-Free Unsupervised Domain Adaptation: A Survey](https://mlsanity.com/paper/2301.00265/) (2022)
- [ChatGPT Based Data Augmentation for Improved Parameter-Efficient Debiasing of LLMs](https://mlsanity.com/paper/2402.11764/) (2024)
- [Automatic Question-Answer Generation for Long-Tail Knowledge](https://mlsanity.com/paper/2403.01382/) (2024)
- [Can You Label Less by Using Out-of-Domain Data? Active & Transfer Learning with Few-shot Instructions](https://mlsanity.com/paper/2211.11798/) (2022)
- [Systematic Assessment of Factual Knowledge in Large Language Models](https://mlsanity.com/paper/2310.11638/) (2023)

## Built on by (most cited first, 19 of 19)

- [TaxoAdapt: Aligning LLM-Based Multidimensional Taxonomy Construction to Evolving Research Corpora](https://mlsanity.com/paper/2506.10737/) (2025)
- [CLLMate: A Multimodal Benchmark for Weather and Climate Events Forecasting](https://mlsanity.com/paper/2409.19058/) (2024)
- [Unmasking Deceptive Visuals: Benchmarking Multimodal Large Language Models on Misleading Chart Question Answering](https://mlsanity.com/paper/2503.18172/) (2025)
- [CodeTaxo: Enhancing Taxonomy Expansion with Limited Examples via Code Language Prompts](https://mlsanity.com/paper/2408.09070/) (2024)
- [Large Language Models for Scholarly Ontology Generation: An Extensive Analysis in the Engineering Field](https://mlsanity.com/paper/2412.08258/) (2024)
- [You Are What You Bought: Generating Customer Personas for E-commerce Applications](https://mlsanity.com/paper/2504.17304/) (2025)
- [OntoURL: A Benchmark for Evaluating Large Language Models on Symbolic Ontological Understanding, Reasoning and Learning](https://mlsanity.com/paper/2505.11031/) (2025)
- [Refining Wikidata Taxonomy using Large Language Models](https://mlsanity.com/paper/2409.04056/) (2024)
- [KERAG: Knowledge-Enhanced Retrieval-Augmented Generation for Advanced Question Answering](https://mlsanity.com/paper/2509.04716/) (2025)
- [Beyond True or False: Retrieval-Augmented Hierarchical Analysis of Nuanced Claims](https://mlsanity.com/paper/2506.10728/) (2025)
- [LoopServe: An Adaptive Dual-phase LLM Inference Acceleration System for Multi-Turn Dialogues](https://mlsanity.com/paper/2507.13681/) (2025)
- [Leveraging Large Language Models for Generating Research Topic Ontologies: A Multi-Disciplinary Study](https://mlsanity.com/paper/2508.20693/) (2025)
- [Enriching Taxonomies Using Large Language Models](https://mlsanity.com/paper/2602.22213/) (2025)
- [GRAVITY: Architecture-Agnostic Structured Anchoring for Long-Horizon Conversational Memory](https://mlsanity.com/paper/2605.01688/) (2026)
- [The Personalization Mirage: How LLMs Fabricate User Profiles, and Why Self-Monitoring Misleads](https://mlsanity.com/paper/2608.04570/) (2026)
- [NeurOWL: An LLM-Based Neural-symbolic Framework for Incomplete OWL Ontology Reasoning](https://mlsanity.com/paper/2607.15776/) (2026)
- [Benchmarking Resource-Efficient LLMs for Research Topic Ontology Generation in the Biomedical Field](https://mlsanity.com/paper/2607.17902/) (2026)
- [ReLTEx: Reliable LLM-based Taxonomy Expansion](https://mlsanity.com/paper/2608.10970/) (2026)
- [WiCleanData: Guaranteeing the Type Consistency of Wikidata by Taxonomy Refinement and Constraint Enforcement](https://mlsanity.com/paper/2609.20057/) (2026)

## Abstract (arXiv)

Large language models (LLMs) demonstrate an impressive ability to internalize knowledge and answer natural language questions. Although previous studies validate that LLMs perform well on general knowledge while presenting poor performance on long-tail nuanced knowledge, the community is still doubtful about whether the traditional knowledge graphs should be replaced by LLMs. In this paper, we ask if the schema of knowledge graph (i.e., taxonomy) is made obsolete by LLMs. Intuitively, LLMs should perform well on common taxonomies and at taxonomy levels that are common to people. Unfortunately, there lacks a comprehensive benchmark that evaluates the LLMs over a wide range of taxonomies from common to specialized domains and at levels from root to leaf so that we can draw a confident conclusion. To narrow the research gap, we constructed a novel taxonomy hierarchical structure discovery benchmark named TaxoGlimpse to evaluate the performance of LLMs over taxonomies. TaxoGlimpse covers ten representative taxonomies from common to specialized domains with in-depth experiments of different levels of entities in this taxonomy from root to leaf. Our comprehensive experiments of eighteen state-of-the-art LLMs under three prompting settings validate that LLMs can still not well capture the knowledge of specialized taxonomies and leaf-level entities.