# Knowledge Tagging with Large Language Model based Multi-Agent System

arXiv: https://arxiv.org/abs/2409.08406 · HTML: https://mlsanity.com/paper/2409.08406/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 12 Sep 2024.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [ToolQA: A Dataset for LLM Question Answering with External Tools](https://mlsanity.com/paper/2306.13304/) (2023)
- [Large Language Models for Education: A Survey and Outlook](https://mlsanity.com/paper/2403.18105/) (2024)
- [Simulating Classroom Education with LLM-Empowered Agents](https://mlsanity.com/paper/2406.19226/) (2024)
- [EKT: Exercise-aware Knowledge Tracing for Student Performance Prediction](https://mlsanity.com/paper/1906.05658/) (2019)
- [Neural Cognitive Diagnosis for Intelligent Education Systems](https://mlsanity.com/paper/1908.08733/) (2019)
- [Evaluating Language Models for Mathematics through Interactions](https://mlsanity.com/paper/2306.01694/) (2023)
- [Bringing Generative AI to Adaptive Learning in Education](https://mlsanity.com/paper/2402.14601/) (2024)
- [QuesNet: A Unified Representation for Heterogeneous Test Questions](https://mlsanity.com/paper/1905.10949/) (2019)
- [DOCENT: Learning Self-Supervised Entity Representations from Large Document Collections](https://mlsanity.com/paper/2102.13247/) (2021)
- [Foundation Models for Education: Promises and Prospects](https://mlsanity.com/paper/2405.10959/) (2024)
- [Knowledge Tagging System on Math Questions via LLMs with Flexible Demonstration Retriever](https://mlsanity.com/paper/2406.13885/) (2024)

## Built on by (most cited first, 5 of 5)

- [A Systematic Assessment of OpenAI o1-Preview for Higher Order Thinking in Education](https://mlsanity.com/paper/2410.21287/) (2024)
- [CAIM: Development and Evaluation of a Cognitive AI Memory Framework for Long-Term Interaction with Intelligent Agents](https://mlsanity.com/paper/2505.13044/) (2025)
- [AgentDR: Dynamic Recommendation with Implicit Item-Item Relations via LLM-based Agents](https://mlsanity.com/paper/2510.05598/) (2025)
- [MAGMA-Edu: Multi-Agent Generative Multimodal Framework for Text-Diagram Educational Question Generation](https://mlsanity.com/paper/2511.18714/) (2025)
- [SRJudge: Empowering Large Language Models with Selective Reasoning for Fine-Grained Knowledge Concept Tagging](https://mlsanity.com/paper/2609.36982/) (2026)

## Abstract (arXiv)

Knowledge tagging for questions is vital in modern intelligent educational applications, including learning progress diagnosis, practice question recommendations, and course content organization. Traditionally, these annotations have been performed by pedagogical experts, as the task demands not only a deep semantic understanding of question stems and knowledge definitions but also a strong ability to link problem-solving logic with relevant knowledge concepts. With the advent of advanced natural language processing (NLP) algorithms, such as pre-trained language models and large language models (LLMs), pioneering studies have explored automating the knowledge tagging process using various machine learning models. In this paper, we investigate the use of a multi-agent system to address the limitations of previous algorithms, particularly in handling complex cases involving intricate knowledge definitions and strict numerical constraints. By demonstrating its superior performance on the publicly available math question knowledge tagging dataset, MathKnowCT, we highlight the significant potential of an LLM-based multi-agent system in overcoming the challenges that previous methods have encountered. Finally, through an in-depth discussion of the implications of automating knowledge tagging, we underscore the promising results of deploying LLM-based algorithms in educational contexts.