# TaxoLLaMA: WordNet-based Model for Solving Multiple Lexical Semantic Tasks

arXiv: https://arxiv.org/abs/2403.09207 · HTML: https://mlsanity.com/paper/2403.09207/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 47 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Builds on (most cited first, 24 of 24)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [Extreme Compression of Large Language Models via Additive Quantization](https://mlsanity.com/paper/2401.06118/) (2024)
- [Unlearn What You Want to Forget: Efficient Unlearning for LLMs](https://mlsanity.com/paper/2310.20150/) (2023)
- [Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networks](https://mlsanity.com/paper/1909.09586/) (2019)
- [Event knowledge in large language models: the gap between the impossible and the unlikely](https://mlsanity.com/paper/2212.01488/) (2022)
- [HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment](https://mlsanity.com/paper/1608.02117/) (2016)
- [Every child should have parents: a taxonomy refinement algorithm based on hyperbolic term embeddings](https://mlsanity.com/paper/1906.02002/) (2019)
- [Taxonomy Completion via Triplet Matching Network](https://mlsanity.com/paper/2101.01896/) (2021)
- [TaxoEnrich: Self-Supervised Taxonomy Completion via Structure-Semantic Representations](https://mlsanity.com/paper/2202.04887/) (2022)
- [Enhancing Taxonomy Completion with Concept Generation via Fusing Relational Representations](https://mlsanity.com/paper/2106.02974/) (2021)
- [Specialising Word Vectors for Lexical Entailment](https://mlsanity.com/paper/1710.06371/) (2017)
- [Probabilistic Models of Relational Implication](https://mlsanity.com/paper/1907.12048/) (2019)
- [pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference](https://mlsanity.com/paper/1810.08854/) (2018)
- [What Do Llamas Really Think? Revealing Preference Biases in Language Model Representations](https://mlsanity.com/paper/2311.18812/) (2023)
- [Distilling Hypernymy Relations from Language Models: On the Effectiveness of Zero-Shot Taxonomy Induction](https://mlsanity.com/paper/2202.04876/) (2022)
- [Smoothing Entailment Graphs with Language Models](https://mlsanity.com/paper/2208.00318/) (2022)
- [SeVeN: Augmenting Word Embeddings with Unsupervised Relation Vectors](https://mlsanity.com/paper/1808.06068/) (2018)

## Built on by (most cited first, 14 of 14)

- [Do LLMs Really Adapt to Domains? An Ontology Learning Perspective](https://mlsanity.com/paper/2407.19998/) (2024)
- [CodeTaxo: Enhancing Taxonomy Expansion with Limited Examples via Code Language Prompts](https://mlsanity.com/paper/2408.09070/) (2024)
- [The LLM Bottleneck: Why Open-Source Vision LLMs Struggle with Hierarchical Visual Recognition](https://mlsanity.com/paper/2505.24840/) (2025)
- [Vision-and-Language Training Helps Deploy Taxonomic Knowledge but Does Not Fundamentally Alter It](https://mlsanity.com/paper/2507.13328/) (2025)
- [Frame Representation Hypothesis: Multi-Token LLM Interpretability and Concept-Guided Text Generation](https://mlsanity.com/paper/2412.07334/) (2024)
- [Talking the Talk Does Not Entail Walking the Walk: On the Limits of Large Language Models in Lexical Entailment Recognition](https://mlsanity.com/paper/2406.14894/) (2024)
- [Meaning Beyond Truth Conditions: Evaluating Discourse Level Understanding via Anaphora Accessibility](https://mlsanity.com/paper/2502.14119/) (2025)
- [KnowCoder-V2: Deep Knowledge Analysis](https://mlsanity.com/paper/2506.06881/) (2025)
- [GEAR: A Simple GENERATE, EMBED, AVERAGE AND RANK Approach for Unsupervised Reverse Dictionary](https://mlsanity.com/paper/2412.06654/) (2024)
- [Do I look like a `cat.n.01` to you? A Taxonomy Image Generation Benchmark](https://mlsanity.com/paper/2503.10357/) (2025)
- [CEFR-Annotated WordNet: LLM-Based Proficiency-Guided Semantic Database for Language Learning](https://mlsanity.com/paper/2510.18466/) (2025)
- [Evontree: Ontology Rule-Guided Self-Evolution of Large Language Models](https://mlsanity.com/paper/2510.26683/) (2025)
- [OpenGloss: A Synthetic Encyclopedic Dictionary and Semantic Knowledge Graph](https://mlsanity.com/paper/2511.18622/) (2025)
- [BEATS: Bootstrapping E-commerce Attribute Taxonomies for Search through Iterative Human-AI Collaboration](https://mlsanity.com/paper/2606.04909/) (2026)

## Abstract (arXiv)

In this paper, we explore the capabilities of LLMs in capturing lexical-semantic knowledge from WordNet on the example of the LLaMA-2-7b model and test it on multiple lexical semantic tasks. As the outcome of our experiments, we present TaxoLLaMA, the everything-in-one model, lightweight due to 4-bit quantization and LoRA. It achieves 11 SotA results, 4 top-2 results out of 16 tasks for the Taxonomy Enrichment, Hypernym Discovery, Taxonomy Construction, and Lexical Entailment tasks. Moreover, it demonstrates very strong zero-shot performance on Lexical Entailment and Taxonomy Construction with no fine-tuning. We also explore its hidden multilingual and domain adaptation capabilities with a little tuning or few-shot learning. All datasets, code, and model are available online at https://github.com/VityaVitalich/TaxoLLaMA