Can LLMs Reason About Attention? Towards Zero-Shot Analysis of Multimodal Classroom Behavior
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Understanding student engagement usually requires time-consuming manual observation or invasive recording that raises privacy concerns. We present a privacy-preserving pipeline that analyzes classroom videos to extract insights about student attention, without storing any identifiable footage. Our system runs on a single GPU, using OpenPose for skeletal extraction and Gaze-LLE for visual attention estimation. Original video frames are deleted immediately after pose extraction, thus only geometric coordinates (stored as JSON) are retained, ensuring compliance with FERPA. The extracted pose and gaze data is processed by QwQ-32B-Reasoning, which performs zero-shot analysis of student behavior across lecture segments. Instructors access results through a web dashboard featuring attention heatmaps and behavioral summaries. Our preliminary findings suggest that LLMs may show promise for multimodal behavior understanding, although they still struggle with spatial reasoning about classroom layouts. We discuss these limitations and outline directions for improving LLM spatial comprehension in educational analytics contexts.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 8 of its 20 references that are in the map. First posted on arXiv 3 Apr 2026.
Builds on
8 of 20 references.
References · 8
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
- 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
- 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
- 2024Gaze-LLE: Gaze Target Estimation via Large-Scale Learned Encoderscited by 36
- 2024Evaluating Large Language Models in Analysing Classroom Dialoguecited by 19
- 2024LLM-Driven Learning Analytics Dashboard for Teachers in EFL Writing Educationcited by 5
- 2024Adversary-Guided Motion Retargeting for Skeleton Anonymizationcited by 1
- 2025Multi-Model Synthetic Training for Mission-Critical Small Language Modelscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Context Matters: Peer-Aware Student Behavioral Engagement Measurement via VLM Action Parsing and LLM Sequence Classification
- 2026Leveraging Vision-Language Models to Detect Attention in Educational Videos
- 2026Zero-Shot Vision-Language Models for Classroom Engagement Recognition: A Benchmark Study of Prompt Sensitivity and Cross-Dataset Generalization
- 2025Exploring Automated Recognition of Instructional Activity and Discourse from Multimodal Classroom Data
- 2026Can Multimodal LLMs See Science Instruction? Benchmarking Pedagogical Reasoning in K-12 Classroom Videos
- 2024Leveraging Language Models for Emotion and Behavior Analysis in Education
- 2025Using Vision Language Models to Detect Students' Academic Emotion through Facial Expressions
- 2025EgoInstruct: An Egocentric Video Dataset of Face-to-face Instructional Interactions with Multi-modal LLM Benchmarking