# TraM : Enhancing User Sleep Prediction with Transformer-based Multivariate Time Series Modeling and Machine Learning Ensembles

arXiv: https://arxiv.org/abs/2410.11293 · HTML: https://mlsanity.com/paper/2410.11293/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 26 references that are in the map.
- First posted on arXiv 15 Oct 2024.

## Builds on (most cited first, 5 of 5)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2306.09364/) (2023)
- [Human Understanding AI Paper Challenge 2024 -- Dataset Design](https://mlsanity.com/paper/2403.16509/) (2024)

## Built on by (most cited first, 3 of 3)

- [Understanding Human Daily Experience Through Continuous Sensing: ETRI Lifelog Dataset 2024](https://mlsanity.com/paper/2508.03698/) (2025)
- [MIS-LSTM: Multichannel Image-Sequence LSTM for Sleep Quality and Stress Prediction](https://mlsanity.com/paper/2509.11232/) (2025)
- [Subject-Adaptive Sparse Linear Models for Interpretable Personalized Health Prediction from Multimodal Lifelog Data](https://mlsanity.com/paper/2510.02835/) (2025)

## Abstract (arXiv)

This paper presents a novel approach that leverages Transformer-based multivariate time series model and Machine Learning Ensembles to predict the quality of human sleep, emotional states, and stress levels. A formula to calculate the labels was developed, and the various models were applied to user data. Time Series Transformer was used for labels where time series characteristics are crucial, while Machine Learning Ensembles were employed for labels requiring comprehensive daily activity statistics. Time Series Transformer excels in capturing the characteristics of time series through pre-training, while Machine Learning Ensembles select machine learning models that meet our categorization criteria. The proposed model, TraM, scored 6.10 out of 10 in experiments, demonstrating superior performance compared to other methodologies. The code and configuration for the TraM framework are available at: https://github.com/jin-jae/ETRI-Paper-Contest.