# Multiple Appropriate Facial Reaction Generation in Dyadic Interaction Settings: What, Why and How?

arXiv: https://arxiv.org/abs/2302.06514 · HTML: https://mlsanity.com/paper/2302.06514/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 28 references that are in the map.
- First posted on arXiv 13 Feb 2023.

## Builds on (most cited first, 8 of 8)

- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motion](https://mlsanity.com/paper/2204.08451/) (2022)
- [Gesticulator: A framework for semantically-aware speech-driven gesture generation](https://mlsanity.com/paper/2001.09326/) (2020)
- [Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition](https://mlsanity.com/paper/2205.01782/) (2022)
- [Exploiting Persona Information for Diverse Generation of Conversational Responses](https://mlsanity.com/paper/1905.12188/) (2019)
- [The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation](https://mlsanity.com/paper/2208.10441/) (2022)
- [Context-Aware Personality Inference in Dyadic Scenarios: Introducing the UDIVA Dataset](https://mlsanity.com/paper/2012.14259/) (2020)
- [GRATIS: Deep Learning Graph Representation with Task-specific Topology and Multi-dimensional Edge Features](https://mlsanity.com/paper/2211.12482/) (2022)

## Built on by (most cited first, 16 of 16)

- [ReactFace: Online Multiple Appropriate Facial Reaction Generation in Dyadic Interactions](https://mlsanity.com/paper/2305.15748/) (2023)
- [REACT 2024: the Second Multiple Appropriate Facial Reaction Generation Challenge](https://mlsanity.com/paper/2401.05166/) (2024)
- [Reversible Graph Neural Network-based Reaction Distribution Learning for Multiple Appropriate Facial Reactions Generation](https://mlsanity.com/paper/2305.15270/) (2023)
- [REACT 2025: the Third Multiple Appropriate Facial Reaction Generation Challenge](https://mlsanity.com/paper/2505.17223/) (2025)
- [REACT2023: the first Multi-modal Multiple Appropriate Facial Reaction Generation Challenge](https://mlsanity.com/paper/2306.06583/) (2023)
- [OmniResponse: Online Multimodal Conversational Response Generation in Dyadic Interactions](https://mlsanity.com/paper/2505.21724/) (2025)
- [Robust Facial Reactions Generation: An Emotion-Aware Framework with Modality Compensation](https://mlsanity.com/paper/2407.15798/) (2024)
- [ReactDiff: Fundamental Multiple Appropriate Facial Reaction Diffusion Model](https://mlsanity.com/paper/2510.04712/) (2025)
- [Latent Behavior Diffusion for Sequential Reaction Generation in Dyadic Setting](https://mlsanity.com/paper/2505.07901/) (2025)
- [MRecGen: Multimodal Appropriate Reaction Generator](https://mlsanity.com/paper/2307.02609/) (2023)
- [Multimodal Quantitative Measures for Multiparty Behaviour Evaluation](https://mlsanity.com/paper/2508.10916/) (2025)
- [PainDiffusion: Learning to Express Pain](https://mlsanity.com/paper/2409.11635/) (2024)
- [REACT 2026: The Fourth Multiple Appropriate Facial Reaction Generation Challenge: Personalised MAFRG and Appropriate EEG Reaction Prediction](https://mlsanity.com/paper/2606.07935/) (2026)
- [ResDiffFRG: Residual Diffusion for Multiple Appropriate Facial Reaction Generation](https://mlsanity.com/paper/2609.33749/) (2026)
- [Beyond End-to-End Black Box Mapping: An Intentional Agent Framework for Cognitive-driven Facial Reaction Generation](https://mlsanity.com/paper/2609.34419/) (2026)
- [GLARE: Generating Listening Heads with Appropriate Reactions](https://mlsanity.com/paper/2609.40317/) (2026)

## Abstract (arXiv)

According to the Stimulus Organism Response (SOR) theory, all human behavioral reactions are stimulated by context, where people will process the received stimulus and produce an appropriate reaction. This implies that in a specific context for a given input stimulus, a person can react differently according to their internal state and other contextual factors. Analogously, in dyadic interactions, humans communicate using verbal and nonverbal cues, where a broad spectrum of listeners' non-verbal reactions might be appropriate for responding to a specific speaker behaviour. There already exists a body of work that investigated the problem of automatically generating an appropriate reaction for a given input. However, none attempted to automatically generate multiple appropriate reactions in the context of dyadic interactions and evaluate the appropriateness of those reactions using objective measures. This paper starts by defining the facial Multiple Appropriate Reaction Generation (fMARG) task for the first time in the literature and proposes a new set of objective evaluation metrics to evaluate the appropriateness of the generated reactions. The paper subsequently introduces a framework to predict, generate, and evaluate multiple appropriate facial reactions.