# MUSE: Multi-Scale Temporal Features Evolution for Knowledge Tracing

arXiv: https://arxiv.org/abs/2102.00228 · HTML: https://mlsanity.com/paper/2102.00228/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 19 references that are in the map.
- First posted on arXiv 30 Jan 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [FreeLB: Enhanced Adversarial Training for Natural Language Understanding](https://mlsanity.com/paper/1909.11764/) (2019)
- [A Self-Attentive model for Knowledge Tracing](https://mlsanity.com/paper/1907.06837/) (2019)
- [EdNet: A Large-Scale Hierarchical Dataset in Education](https://mlsanity.com/paper/1912.03072/) (2019)
- [Towards an Appropriate Query, Key, and Value Computation for Knowledge Tracing](https://mlsanity.com/paper/2002.07033/) (2020)
- [SAINT+: Integrating Temporal Features for EdNet Correctness Prediction](https://mlsanity.com/paper/2010.12042/) (2020)
- [Multi-Scale Self-Attention for Text Classification](https://mlsanity.com/paper/1912.00544/) (2019)
- [MRIF: Multi-resolution Interest Fusion for Recommendation](https://mlsanity.com/paper/2007.07084/) (2020)

## Built on by (most cited first, 3 of 3)

- [Assessing the Performance of Online Students -- New Data, New Approaches, Improved Accuracy](https://mlsanity.com/paper/2109.01753/) (2021)
- [Multi-granulariy Time-based Transformer for Knowledge Tracing](https://mlsanity.com/paper/2304.05257/) (2023)
- [Improved Sales Forecasting using Trend and Seasonality Decomposition with LightGBM](https://mlsanity.com/paper/2305.17201/) (2023)

## Abstract (arXiv)

Transformer based knowledge tracing model is an extensively studied problem in the field of computer-aided education. By integrating temporal features into the encoder-decoder structure, transformers can processes the exercise information and student response information in a natural way. However, current state-of-the-art transformer-based variants still share two limitations. First, extremely long temporal features cannot well handled as the complexity of self-attention mechanism is O(n2). Second, existing approaches track the knowledge drifts under fixed a window size, without considering different temporal-ranges. To conquer these problems, we propose MUSE, which is equipped with multi-scale temporal sensor unit, that takes either local or global temporal features into consideration. The proposed model is capable to capture the dynamic changes in users knowledge states at different temporal-ranges, and provides an efficient and powerful way to combine local and global features to make predictions. Our method won the 5-th place over 3,395 teams in the Riiid AIEd Challenge 2020.