# Multi-Modal Machine Learning for Assessing Gaming Skills in Online Streaming: A Case Study with CS:GO

arXiv: https://arxiv.org/abs/2307.12236 · HTML: https://mlsanity.com/paper/2307.12236/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 23 Jul 2023.

## Builds on (most cited first, 23 of 23)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [Multimodal Transformer for Unaligned Multimodal Language Sequences](https://mlsanity.com/paper/1906.00295/) (2019)
- [SoundNet: Learning Sound Representations from Unlabeled Video](https://mlsanity.com/paper/1610.09001/) (2016)
- [Beyond Short Snippets: Deep Networks for Video Classification](https://mlsanity.com/paper/1503.08909/) (2015)
- [Focal Modulation Networks](https://mlsanity.com/paper/2203.11926/) (2022)
- [Learning a Text-Video Embedding from Incomplete and Heterogeneous Data](https://mlsanity.com/paper/1804.02516/) (2018)
- [TubeDETR: Spatio-Temporal Video Grounding with Transformers](https://mlsanity.com/paper/2203.16434/) (2022)
- [LipNet: End-to-End Sentence-level Lipreading](https://mlsanity.com/paper/1611.01599/) (2016)
- [Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversations](https://mlsanity.com/paper/2109.12174/) (2021)
- [Complex Dynamic Neurons Improved Spiking Transformer Network for Efficient Automatic Speech Recognition](https://mlsanity.com/paper/2302.01194/) (2023)
- [Video Highlight Prediction Using Audience Chat Reactions](https://mlsanity.com/paper/1707.08559/) (2017)
- [Deep Learning for Genomics: A Concise Overview](https://mlsanity.com/paper/1802.00810/) (2018)
- [Deep Unsupervised Multi-View Detection of Video Game Stream Highlights](https://mlsanity.com/paper/1807.09715/) (2018)
- [Multimodal Joint Emotion and Game Context Recognition in League of Legends Livestreams](https://mlsanity.com/paper/1905.13694/) (2019)
- [Real-Time Video Highlights for Yahoo Esports](https://mlsanity.com/paper/1611.08780/) (2016)
- [Between an Arena and a Sports Bar: Online Chats of eSports Spectators](https://mlsanity.com/paper/1801.02862/) (2018)
- [Integrity and Junkiness Failure Handling for Embedding-based Retrieval: A Case Study in Social Network Search](https://mlsanity.com/paper/2304.09287/) (2023)
- [A Sparse Graph-Structured Lasso Mixed Model for Genetic Association with Confounding Correction](https://mlsanity.com/paper/1711.04162/) (2017)
- [E-Sports Talent Scouting Based on Multimodal Twitch Stream Data](https://mlsanity.com/paper/1907.01615/) (2019)

## Abstract (arXiv)

Online streaming is an emerging market that address much attention. Assessing gaming skills from videos is an important task for streaming service providers to discover talented gamers. Service providers require the information to offer customized recommendation and service promotion to their customers. Meanwhile, this is also an important multi-modal machine learning tasks since online streaming combines vision, audio and text modalities. In this study we begin by identifying flaws in the dataset and proceed to clean it manually. Then we propose several variants of latest end-to-end models to learn joint representation of multiple modalities. Through our extensive experimentation, we demonstrate the efficacy of our proposals. Moreover, we identify that our proposed models is prone to identifying users instead of learning meaningful representations. We purpose future work to address the issue in the end.