# Multimodal Analytics of Cybersecurity Crisis Preparation Exercises: What Predicts Success?

arXiv: https://arxiv.org/abs/2603.28553 · HTML: https://mlsanity.com/paper/2603.28553/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 48 references that are in the map.
- First posted on arXiv 30 Mar 2026.

## Builds on (most cited first, 13 of 13)

- [Evaluating the Explainers: Black-Box Explainable Machine Learning for Student Success Prediction in MOOCs](https://mlsanity.com/paper/2207.00551/) (2022)
- [Text Classification in the LLM Era -- Where do we stand?](https://mlsanity.com/paper/2502.11830/) (2025)
- [Predicting challenge moments from students' discourse: A comparison of GPT-4 to two traditional natural language processing approaches](https://mlsanity.com/paper/2401.01692/) (2024)
- [Disentangling Learning from Judgment: Representation Learning for Open Response Analytics](https://mlsanity.com/paper/2512.23941/) (2025)
- [Scalable Learning Environments for Teaching Cybersecurity Hands-on](https://mlsanity.com/paper/2110.10004/) (2021)
- [Applications of Educational Data Mining and Learning Analytics on Data From Cybersecurity Training](https://mlsanity.com/paper/2307.08582/) (2023)
- [Research and Practice of Delivering Tabletop Exercises](https://mlsanity.com/paper/2404.10206/) (2024)
- [From Paper to Platform: Evolution of a Novel Learning Environment for Tabletop Exercises](https://mlsanity.com/paper/2404.10988/) (2024)
- [Rethinking the Potential of Multimodality in Collaborative Problem Solving Diagnosis with Large Language Models](https://mlsanity.com/paper/2504.15093/) (2025)
- [What Are Cybersecurity Education Papers About? A Systematic Literature Review of SIGCSE and ITiCSE Conferences](https://mlsanity.com/paper/1911.11675/) (2019)
- [Revealing Networks: Understanding Effective Teacher Practices in AI-Supported Classrooms using Transmodal Ordered Network Analysis](https://mlsanity.com/paper/2312.10826/) (2023)
- [Combining Log Data and Collaborative Dialogue Features to Predict Project Quality in Middle School AI Education](https://mlsanity.com/paper/2506.11326/) (2025)
- [Exploring Student Expectations and Confidence in Learning Analytics](https://mlsanity.com/paper/2601.05082/) (2026)

## Abstract (arXiv)

Instructional alignment, the match between intended cognition and enacted activity, is central to effective instruction but hard to operationalize at scale. We examine alignment in cybersecurity simulations using multimodal traces from 23 teams (76 students) across five exercise sessions. Study 1 codes objectives and team emails with Bloom's taxonomy and models the completion of key exercise tasks with generalized linear mixed models. Alignment, defined as the discrepancy between required and enacted Bloom levels, predicts success, whereas the Bloom category alone does not predict success once discrepancy is considered. Study 2 compares predictive feature families using grouped cross-validation and l1-regularized logistic regression. Text embeddings and log features outperform Bloom-only models (AUC~0.74 and 0.71 vs. 0.55), and their combination performs best (Test AUC~0.80), with Bloom frequencies adding little. Overall, the work offers a measure of alignment for simulations and shows that multimodal traces best forecast performance, while alignment provides interpretable diagnostic insight.