# MONAH: Multi-Modal Narratives for Humans to analyze conversations

arXiv: https://arxiv.org/abs/2101.07339 · HTML: https://mlsanity.com/paper/2101.07339/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 57 references that are in the map.
- First posted on arXiv 18 Jan 2021.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [DialogueRNN: An Attentive RNN for Emotion Detection in Conversations](https://mlsanity.com/paper/1811.00405/) (2018)
- [Memory Fusion Network for Multi-view Sequential Learning](https://mlsanity.com/paper/1802.00927/) (2018)
- [A Theoretically Grounded Application of Dropout in Recurrent Neural Networks](https://mlsanity.com/paper/1512.05287/) (2015)
- [M3ER: Multiplicative Multimodal Emotion Recognition Using Facial, Textual, and Speech Cues](https://mlsanity.com/paper/1911.05659/) (2019)
- [Detecting Low Rapport During Natural Interactions in Small Groups from Non-Verbal Behaviour](https://mlsanity.com/paper/1801.06055/) (2018)
- [A Comparison of Online Automatic Speech Recognition Systems and the Nonverbal Responses to Unintelligible Speech](https://mlsanity.com/paper/1904.12403/) (2019)
- [A Review on Dyadic Conversation Visualizations - Purposes, Data, Lens of Analysis](https://mlsanity.com/paper/1905.00653/) (2019)
- [Detecting depression in dyadic conversations with multimodal narratives and visualizations](https://mlsanity.com/paper/2001.04809/) (2020)

## Built on by (most cited first, 1 of 1)

- [Transfer Learning in Conversational Analysis through Reusing Preprocessing Data as Supervisors](https://mlsanity.com/paper/2112.03032/) (2021)

## Abstract (arXiv)

In conversational analyses, humans manually weave multimodal information into the transcripts, which is significantly time-consuming. We introduce a system that automatically expands the verbatim transcripts of video-recorded conversations using multimodal data streams. This system uses a set of preprocessing rules to weave multimodal annotations into the verbatim transcripts and promote interpretability. Our feature engineering contributions are two-fold: firstly, we identify the range of multimodal features relevant to detect rapport-building; secondly, we expand the range of multimodal annotations and show that the expansion leads to statistically significant improvements in detecting rapport-building.