# DCTM: Dilated Convolutional Transformer Model for Multimodal Engagement Estimation in Conversation

arXiv: https://arxiv.org/abs/2308.01966 · HTML: https://mlsanity.com/paper/2308.01966/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 43 references that are in the map.
- First posted on arXiv 31 Jul 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Attention Bottlenecks for Multimodal Fusion](https://mlsanity.com/paper/2107.00135/) (2021)
- [Gated Multimodal Units for Information Fusion](https://mlsanity.com/paper/1702.01992/) (2017)
- [Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentation](https://mlsanity.com/paper/1805.04574/) (2018)
- [Fusion and Orthogonal Projection for Improved Face-Voice Association](https://mlsanity.com/paper/2112.10483/) (2021)
- [Single-branch Network for Multimodal Training](https://mlsanity.com/paper/2303.06129/) (2023)
- [Learning Branched Fusion and Orthogonal Projection for Face-Voice Association](https://mlsanity.com/paper/2208.10238/) (2022)
- [An Estimation Framework for Passerby Engagement Interacting with Social Robots](https://mlsanity.com/paper/2206.02394/) (2022)

## Built on by (most cited first, 3 of 3)

- [PAEFF: Precise Alignment and Enhanced Gated Feature Fusion for Face-Voice Association](https://mlsanity.com/paper/2505.17002/) (2025)
- [Interpretable Concept-based Deep Learning Framework for Multimodal Human Behavior Modeling](https://mlsanity.com/paper/2502.10145/) (2025)
- [Multi-Domain EEG Representation Learning with Orthogonal Mapping and Attention-based Fusion for Cognitive Load Classification](https://mlsanity.com/paper/2511.12394/) (2025)

## Abstract (arXiv)

Conversational engagement estimation is posed as a regression problem, entailing the identification of the favorable attention and involvement of the participants in the conversation. This task arises as a crucial pursuit to gain insights into human's interaction dynamics and behavior patterns within a conversation. In this research, we introduce a dilated convolutional Transformer for modeling and estimating human engagement in the MULTIMEDIATE 2023 competition. Our proposed system surpasses the baseline models, exhibiting a noteworthy 7% improvement on test set and 4% on validation set. Moreover, we employ different modality fusion mechanism and show that for this type of data, a simple concatenated method with self-attention fusion gains the best performance.