# Class-attention Video Transformer for Engagement Intensity Prediction

arXiv: https://arxiv.org/abs/2208.07216 · HTML: https://mlsanity.com/paper/2208.07216/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 12 Aug 2022.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Going deeper with Image Transformers](https://mlsanity.com/paper/2103.17239/) (2021)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [Vision Transformer for Small-Size Datasets](https://mlsanity.com/paper/2112.13492/) (2021)
- [DAiSEE: Towards User Engagement Recognition in the Wild](https://mlsanity.com/paper/1609.01885/) (2016)
- [Prediction and Localization of Student Engagement in the Wild](https://mlsanity.com/paper/1804.00858/) (2018)
- [Improving state-of-the-art in Detecting Student Engagement with Resnet and TCN Hybrid Network](https://mlsanity.com/paper/2104.10122/) (2021)

## Built on by (most cited first, 7 of 7)

- [Streaming Video Diffusion: Online Video Editing with Diffusion Models](https://mlsanity.com/paper/2405.19726/) (2024)
- [Engagement Measurement Based on Facial Landmarks and Spatial-Temporal Graph Convolutional Networks](https://mlsanity.com/paper/2403.17175/) (2024)
- [Bag of States: A Non-sequential Approach to Video-based Engagement Measurement](https://mlsanity.com/paper/2301.06730/) (2023)
- [Learning in Focus: Detecting Behavioral and Collaborative Engagement Using Vision Transformers](https://mlsanity.com/paper/2508.15782/) (2025)
- [Multipar-T: Multiparty-Transformer for Capturing Contingent Behaviors in Group Conversations](https://mlsanity.com/paper/2304.12204/) (2023)
- [Supervised Contrastive Learning for Ordinal Engagement Measurement](https://mlsanity.com/paper/2505.20676/) (2025)
- [OPEN: A Benchmark Dataset and Baseline for Older Adult Patient Engagement Recognition in Virtual Rehabilitation Learning Environments](https://mlsanity.com/paper/2507.17959/) (2025)

## Abstract (arXiv)

In order to deal with variant-length long videos, prior works extract multi-modal features and fuse them to predict students' engagement intensity. In this paper, we present a new end-to-end method Class Attention in Video Transformer (CavT), which involves a single vector to process class embedding and to uniformly perform end-to-end learning on variant-length long videos and fixed-length short videos. Furthermore, to address the lack of sufficient samples, we propose a binary-order representatives sampling method (BorS) to add multiple video sequences of each video to augment the training set. BorS+CavT not only achieves the state-of-the-art MSE (0.0495) on the EmotiW-EP dataset, but also obtains the state-of-the-art MSE (0.0377) on the DAiSEE dataset. The code and models have been made publicly available at https://github.com/mountainai/cavt.