# PersonaTAB: Predicting Personality Traits using Textual, Acoustic, and Behavioral Cues in Fully-Duplex Speech Dialogs

arXiv: https://arxiv.org/abs/2505.14356 · HTML: https://mlsanity.com/paper/2505.14356/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 39 references that are in the map.
- First posted on arXiv 20 May 2025.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities](https://mlsanity.com/paper/2305.11000/) (2023)
- [LLaMA-Omni: Seamless Speech Interaction with Large Language Models](https://mlsanity.com/paper/2409.06666/) (2024)
- [Generative Spoken Dialogue Language Modeling](https://mlsanity.com/paper/2203.16502/) (2022)
- [Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLM](https://mlsanity.com/paper/2305.15255/) (2023)
- [WavChat: A Survey of Spoken Dialogue Models](https://mlsanity.com/paper/2411.13577/) (2024)
- [Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics](https://mlsanity.com/paper/2503.01174/) (2025)
- [A Full-duplex Speech Dialogue Scheme Based On Large Language Models](https://mlsanity.com/paper/2405.19487/) (2024)
- [Is ChatGPT a Good Personality Recognizer? A Preliminary Study](https://mlsanity.com/paper/2307.03952/) (2023)
- [Automatic Personality Prediction; an Enhanced Method Using Ensemble Modeling](https://mlsanity.com/paper/2007.04571/) (2020)
- [Continuous Output Personality Detection Models via Mixed Strategy Training](https://mlsanity.com/paper/2406.16223/) (2024)
- [Modeling Dyadic Conversations for Personality Inference](https://mlsanity.com/paper/2009.12496/) (2020)
- [Generate labeled training data using Prompt Programming and GPT-3. An example of Big Five Personality Classification](https://mlsanity.com/paper/2303.12279/) (2023)

## Abstract (arXiv)

Despite significant progress in neural spoken dialog systems, personality-aware conversation agents -- capable of adapting behavior based on personalities -- remain underexplored due to the absence of personality annotations in speech datasets. We propose a pipeline that preprocesses raw audio recordings to create a dialogue dataset annotated with timestamps, response types, and emotion/sentiment labels. We employ an automatic speech recognition (ASR) system to extract transcripts and timestamps, then generate conversation-level annotations. Leveraging these annotations, we design a system that employs large language models to predict conversational personality. Human evaluators were engaged to identify conversational characteristics and assign personality labels. Our analysis demonstrates that the proposed system achieves stronger alignment with human judgments compared to existing approaches.