# Learning Structure and Knowledge Aware Representation with Large Language Models for Concept Recommendation

arXiv: https://arxiv.org/abs/2405.12442 · HTML: https://mlsanity.com/paper/2405.12442/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 21 May 2024.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning](https://mlsanity.com/paper/1801.07606/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Self-supervised Graph Learning for Recommendation](https://mlsanity.com/paper/2010.10783/) (2020)
- [Text Is All You Need: Learning Language Representations for Sequential Recommendation](https://mlsanity.com/paper/2305.13731/) (2023)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [pyKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models](https://mlsanity.com/paper/2206.11460/) (2022)
- [Zero-Shot Recommender Systems](https://mlsanity.com/paper/2105.08318/) (2021)
- [Exploiting Cognitive Structure for Adaptive Learning](https://mlsanity.com/paper/1905.12470/) (2019)
- [Attentional Graph Convolutional Networks for Knowledge Concept Recommendation in MOOCs in a Heterogeneous View](https://mlsanity.com/paper/2006.13257/) (2020)

## Built on by (most cited first, 10 of 10)

- [LLM Agents for Education: Advances and Applications](https://mlsanity.com/paper/2503.11733/) (2025)
- [Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyond](https://mlsanity.com/paper/2410.19744/) (2024)
- [Classroom Simulacra: Building Contextual Student Generative Agents in Online Education for Learning Behavioral Simulation](https://mlsanity.com/paper/2502.02780/) (2025)
- [Graph Foundation Models for Recommendation: A Comprehensive Survey](https://mlsanity.com/paper/2502.08346/) (2025)
- [IB-GRPO: Aligning LLM-based Learning Path Recommendation with Educational Objectives via Indicator-Based Group Relative Policy Optimization](https://mlsanity.com/paper/2601.14686/) (2026)
- [Enhancing Learning Path Recommendation via Multi-task Learning](https://mlsanity.com/paper/2507.05295/) (2025)
- [Cognitive-Level Adaptive Generation via Capability-Aware Retrieval and Style Adaptation](https://mlsanity.com/paper/2509.19336/) (2025)
- [CLLMRec: LLM-powered Cognitive-Aware Concept Recommendation via Semantic Alignment and Prerequisite Knowledge Distillation](https://mlsanity.com/paper/2511.17041/) (2025)
- [Incorporating Cognitive Load and Knowledge Transfer for Multi-Domain Knowledge Tracing](https://mlsanity.com/paper/2608.24005/) (2026)
- [TutorLoop: Regulating Student Learning Behaviors via Sensor-in-the-Loop Generative Feedback](https://mlsanity.com/paper/2610.09400/) (2026)

## Abstract (arXiv)

Concept recommendation aims to suggest the next concept for learners to study based on their knowledge states and the human knowledge system. While knowledge states can be predicted using knowledge tracing models, previous approaches have not effectively integrated the human knowledge system into the process of designing these educational models. In the era of rapidly evolving Large Language Models (LLMs), many fields have begun using LLMs to generate and encode text, introducing external knowledge. However, integrating LLMs into concept recommendation presents two urgent challenges: 1) How to construct text for concepts that effectively incorporate the human knowledge system? 2) How to adapt non-smooth, anisotropic text encodings effectively for concept recommendation? In this paper, we propose a novel Structure and Knowledge Aware Representation learning framework for concept Recommendation (SKarREC). We leverage factual knowledge from LLMs as well as the precedence and succession relationships between concepts obtained from the knowledge graph to construct textual representations of concepts. Furthermore, we propose a graph-based adapter to adapt anisotropic text embeddings to the concept recommendation task. This adapter is pre-trained through contrastive learning on the knowledge graph to get a smooth and structure-aware concept representation. Then, it's fine-tuned through the recommendation task, forming a text-to-knowledge-to-recommendation adaptation pipeline, which effectively constructs a structure and knowledge-aware concept representation. Our method does a better job than previous adapters in transforming text encodings for application in concept recommendation. Extensive experiments on real-world datasets demonstrate the effectiveness of the proposed approach.