# Student Classroom Behavior Detection based on Spatio-Temporal Network and Multi-Model Fusion

arXiv: https://arxiv.org/abs/2310.16267 · HTML: https://mlsanity.com/paper/2310.16267/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 34 references that are in the map.
- First posted on arXiv 25 Oct 2023.

## Builds on (most cited first, 19 of 19)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors](https://mlsanity.com/paper/2207.02696/) (2022)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [Masked Autoencoders As Spatiotemporal Learners](https://mlsanity.com/paper/2205.09113/) (2022)
- [CrowdHuman: A Benchmark for Detecting Human in a Crowd](https://mlsanity.com/paper/1805.00123/) (2018)
- [BSN: Boundary Sensitive Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1806.02964/) (2018)
- [Reconstruction Network for Video Captioning](https://mlsanity.com/paper/1803.11438/) (2018)
- [Fast Learning of Temporal Action Proposal via Dense Boundary Generator](https://mlsanity.com/paper/1911.04127/) (2019)
- [A Comprehensive Study of Deep Video Action Recognition](https://mlsanity.com/paper/2012.06567/) (2020)
- [MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions](https://mlsanity.com/paper/2105.07404/) (2021)
- [The AVA-Kinetics Localized Human Actions Video Dataset](https://mlsanity.com/paper/2005.00214/) (2020)
- [Actions as Moving Points](https://mlsanity.com/paper/2001.04608/) (2020)
- [Synergy between 3DMM and 3D Landmarks for Accurate 3D Facial Geometry](https://mlsanity.com/paper/2110.09772/) (2021)
- [Watch, Listen, and Describe: Globally and Locally Aligned Cross-Modal Attentions for Video Captioning](https://mlsanity.com/paper/1804.05448/) (2018)
- [Actor Conditioned Attention Maps for Video Action Detection](https://mlsanity.com/paper/1812.11631/) (2018)
- [Student Classroom Behavior Detection based on YOLOv7-BRA and Multi-Model Fusion](https://mlsanity.com/paper/2305.07825/) (2023)
- [SCB-Dataset3: A Benchmark for Detecting Student Classroom Behavior](https://mlsanity.com/paper/2310.02522/) (2023)
- [Student Classroom Behavior Detection based on Improved YOLOv7](https://mlsanity.com/paper/2306.03318/) (2023)

## Abstract (arXiv)

Using deep learning methods to detect students' classroom behavior automatically is a promising approach for analyzing their class performance and improving teaching effectiveness. However, the lack of publicly available spatio-temporal datasets on student behavior, as well as the high cost of manually labeling such datasets, pose significant challenges for researchers in this field. To address this issue, we proposed a method for extending the spatio-temporal behavior dataset in Student Classroom Scenarios (SCB-ST-Dataset4) through image dataset. Our SCB-ST-Dataset4 comprises 757265 images with 25810 labels, focusing on 3 behaviors: hand-raising, reading, writing. Our proposed method can rapidly generate spatio-temporal behavior datasets without requiring extra manual labeling. Furthermore, we proposed a Behavior Similarity Index (BSI) to explore the similarity of behaviors. We evaluated the dataset using the YOLOv5, YOLOv7, YOLOv8, and SlowFast algorithms, achieving a mean average precision (map) of up to 82.3%. Last, we fused multiple models to generate student behavior-related data from various perspectives. The experiment further demonstrates the effectiveness of our method. And SCB-ST-Dataset4 provides a robust foundation for future research in student behavior detection, potentially contributing to advancements in this field. The SCB-ST-Dataset4 is available for download at: https://github.com/Whiffe/SCB-dataset.