# REACT2023: the first Multi-modal Multiple Appropriate Facial Reaction Generation Challenge

arXiv: https://arxiv.org/abs/2306.06583 · HTML: https://mlsanity.com/paper/2306.06583/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 46 references that are in the map.
- First posted on arXiv 11 Jun 2023.

## Builds on (most cited first, 30 of 30)

- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [Diffusion Autoencoders: Toward a Meaningful and Decodable Representation](https://mlsanity.com/paper/2111.15640/) (2021)
- [FaceFormer: Speech-Driven 3D Facial Animation with Transformers](https://mlsanity.com/paper/2112.05329/) (2021)
- [PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering](https://mlsanity.com/paper/2109.08379/) (2021)
- [MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement](https://mlsanity.com/paper/2104.08223/) (2021)
- [DLow: Diversifying Latent Flows for Diverse Human Motion Prediction](https://mlsanity.com/paper/2003.08386/) (2020)
- [CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior](https://mlsanity.com/paper/2301.02379/) (2023)
- [TEACH: Temporal Action Composition for 3D Humans](https://mlsanity.com/paper/2209.04066/) (2022)
- [GANimation: Anatomically-aware Facial Animation from a Single Image](https://mlsanity.com/paper/1807.09251/) (2018)
- [Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motion](https://mlsanity.com/paper/2204.08451/) (2022)
- [FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Dataset](https://mlsanity.com/paper/2203.14057/) (2022)
- [TorchAudio: Building Blocks for Audio and Speech Processing](https://mlsanity.com/paper/2110.15018/) (2021)
- [Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition](https://mlsanity.com/paper/2205.01782/) (2022)
- [BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Prediction](https://mlsanity.com/paper/2211.14304/) (2022)
- [Responsive Listening Head Generation: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2112.13548/) (2021)
- [Let's Face It: Probabilistic Multi-modal Interlocutor-aware Generation of Facial Gestures in Dyadic Settings](https://mlsanity.com/paper/2006.09888/) (2020)
- [Affective Faces for Goal-Driven Dyadic Communication](https://mlsanity.com/paper/2301.10939/) (2023)
- [Context-Aware Personality Inference in Dyadic Scenarios: Introducing the UDIVA Dataset](https://mlsanity.com/paper/2012.14259/) (2020)
- [Dynamic Facial Expression Generation on Hilbert Hypersphere with Conditional Wasserstein Generative Adversarial Nets](https://mlsanity.com/paper/1907.10087/) (2019)
- [Multiple Appropriate Facial Reaction Generation in Dyadic Interaction Settings: What, Why and How?](https://mlsanity.com/paper/2302.06514/) (2023)
- [Interactive Generative Adversarial Networks for Facial Expression Generation in Dyadic Interactions](https://mlsanity.com/paper/1801.09092/) (2018)
- [Joint Audio-Text Model for Expressive Speech-Driven 3D Facial Animation](https://mlsanity.com/paper/2112.02214/) (2021)
- [Didn't see that coming: a survey on non-verbal social human behavior forecasting](https://mlsanity.com/paper/2203.02480/) (2022)
- [GRATIS: Deep Learning Graph Representation with Task-specific Topology and Multi-dimensional Edge Features](https://mlsanity.com/paper/2211.12482/) (2022)
- [Reversible Graph Neural Network-based Reaction Distribution Learning for Multiple Appropriate Facial Reactions Generation](https://mlsanity.com/paper/2305.15270/) (2023)
- [Comparison of Spatio-Temporal Models for Human Motion and Pose Forecasting in Face-to-Face Interaction Scenarios](https://mlsanity.com/paper/2203.03245/) (2022)
- [A Realistic Face-to-Face Conversation System based on Deep Neural Networks](https://mlsanity.com/paper/1908.07750/) (2019)
- [AMII: Adaptive Multimodal Inter-personal and Intra-personal Model for Adapted Behavior Synthesis](https://mlsanity.com/paper/2305.11310/) (2023)

## Built on by (most cited first, 5 of 5)

- [Beyond Talking -- Generating Holistic 3D Human Dyadic Motion for Communication](https://mlsanity.com/paper/2403.19467/) (2024)
- [Robust Facial Reactions Generation: An Emotion-Aware Framework with Modality Compensation](https://mlsanity.com/paper/2407.15798/) (2024)
- [Latent Behavior Diffusion for Sequential Reaction Generation in Dyadic Setting](https://mlsanity.com/paper/2505.07901/) (2025)
- [MRecGen: Multimodal Appropriate Reaction Generator](https://mlsanity.com/paper/2307.02609/) (2023)
- [ReactDiff: Latent Diffusion for Facial Reaction Generation](https://mlsanity.com/paper/2505.14151/) (2025)

## Abstract (arXiv)

The Multi-modal Multiple Appropriate Facial Reaction Generation Challenge (REACT2023) is the first competition event focused on evaluating multimedia processing and machine learning techniques for generating human-appropriate facial reactions in various dyadic interaction scenarios, with all participants competing strictly under the same conditions. The goal of the challenge is to provide the first benchmark test set for multi-modal information processing and to foster collaboration among the audio, visual, and audio-visual affective computing communities, to compare the relative merits of the approaches to automatic appropriate facial reaction generation under different spontaneous dyadic interaction conditions. This paper presents: (i) novelties, contributions and guidelines of the REACT2023 challenge; (ii) the dataset utilized in the challenge; and (iii) the performance of baseline systems on the two proposed sub-challenges: Offline Multiple Appropriate Facial Reaction Generation and Online Multiple Appropriate Facial Reaction Generation, respectively. The challenge baseline code is publicly available at urlhttps://github.com/reactmultimodalchallenge/baselinereact2023.