# Transfer Learning in Conversational Analysis through Reusing Preprocessing Data as Supervisors

arXiv: https://arxiv.org/abs/2112.03032 · HTML: https://mlsanity.com/paper/2112.03032/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 82 references that are in the map.
- First posted on arXiv 2 Dec 2021.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [Deep Double Descent: Where Bigger Models and More Data Hurt](https://mlsanity.com/paper/1912.02292/) (2019)
- [DialogueRNN: An Attentive RNN for Emotion Detection in Conversations](https://mlsanity.com/paper/1811.00405/) (2018)
- [Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks](https://mlsanity.com/paper/1911.07883/) (2019)
- [Memory Fusion Network for Multi-view Sequential Learning](https://mlsanity.com/paper/1802.00927/) (2018)
- [Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances](https://mlsanity.com/paper/1905.02947/) (2019)
- [Understanding and Improving Information Transfer in Multi-Task Learning](https://mlsanity.com/paper/2005.00944/) (2020)
- [Learning to Diagnose with LSTM Recurrent Neural Networks](https://mlsanity.com/paper/1511.03677/) (2015)
- [Multi-attention Recurrent Network for Human Communication Comprehension](https://mlsanity.com/paper/1802.00923/) (2018)
- [Unsupervised Multi-Task Feature Learning on Point Clouds](https://mlsanity.com/paper/1910.08207/) (2019)
- [Learning Longer-term Dependencies in RNNs with Auxiliary Losses](https://mlsanity.com/paper/1803.00144/) (2018)
- [M3ER: Multiplicative Multimodal Emotion Recognition Using Facial, Textual, and Speech Cues](https://mlsanity.com/paper/1911.05659/) (2019)
- [Knowledge Transfer for Melanoma Screening with Deep Learning](https://mlsanity.com/paper/1703.07479/) (2017)
- [Deep Asymmetric Multi-task Feature Learning](https://mlsanity.com/paper/1708.00260/) (2017)
- [Multi-Task Semi-Supervised Adversarial Autoencoding for Speech Emotion Recognition](https://mlsanity.com/paper/1907.06078/) (2019)
- [Hierarchical Multitask Learning for CTC-based Speech Recognition](https://mlsanity.com/paper/1807.06234/) (2018)
- [Adversarial Multitask Learning for Joint Multi-Feature and Multi-Dialect Morphological Modeling](https://mlsanity.com/paper/1910.12702/) (2019)
- [Deeper Image Quality Transfer: Training Low-Memory Neural Networks for 3D Images](https://mlsanity.com/paper/1808.05577/) (2018)
- [Auxiliary-task Based Deep Reinforcement Learning for Participant Selection Problem in Mobile Crowdsourcing](https://mlsanity.com/paper/2008.11087/) (2020)
- [A Review on Dyadic Conversation Visualizations - Purposes, Data, Lens of Analysis](https://mlsanity.com/paper/1905.00653/) (2019)
- [Multitask Learning for Blackmarket Tweet Detection](https://mlsanity.com/paper/1907.04072/) (2019)
- [Detecting depression in dyadic conversations with multimodal narratives and visualizations](https://mlsanity.com/paper/2001.04809/) (2020)
- [MONAH: Multi-Modal Narratives for Humans to analyze conversations](https://mlsanity.com/paper/2101.07339/) (2021)

## Abstract (arXiv)

Conversational analysis systems are trained using noisy human labels and often require heavy preprocessing during multi-modal feature extraction. Using noisy labels in single-task learning increases the risk of over-fitting. Auxiliary tasks could improve the performance of the primary task learning during the same training -- this approach sits in the intersection of transfer learning and multi-task learning (MTL). In this paper, we explore how the preprocessed data used for feature engineering can be re-used as auxiliary tasks, thereby promoting the productive use of data. Our main contributions are: (1) the identification of sixteen beneficially auxiliary tasks, (2) studying the method of distributing learning capacity between the primary and auxiliary tasks, and (3) studying the relative supervision hierarchy between the primary and auxiliary tasks. Extensive experiments on IEMOCAP and SEMAINE data validate the improvements over single-task approaches, and suggest that it may generalize across multiple primary tasks.