# Speech Signals Complement LLMs for Predicting Interpersonal Attraction in Speed Dating

arXiv: https://arxiv.org/abs/2607.23037 · HTML: https://mlsanity.com/paper/2607.23037/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 41 references that are in the map.
- First posted on arXiv 25 Jul 2026.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Libri-Light: A Benchmark for ASR with Limited or No Supervision](https://mlsanity.com/paper/1912.07875/) (2019)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2410.19168/) (2024)
- [When Large Language Models are Reliable for Judging Empathic Communication](https://mlsanity.com/paper/2506.10150/) (2025)
- [Using LLMs for Late Multimodal Sensor Fusion for Activity Recognition](https://mlsanity.com/paper/2509.10729/) (2025)
- [Multimodal Fusion with LLMs for Engagement Prediction in Natural Conversation](https://mlsanity.com/paper/2409.09135/) (2024)

## Abstract (arXiv)

Large language models (LLMs) can predict interpersonal attraction from conversation transcripts, but it remains unclear what a speech predictor can add beyond transcript-only LLM prediction. Using Japanese speed-dating conversations, we combine predictions from a transcript-only LLM and a supervised speech predictor to estimate participants' reported liking of their partners. We show that speech can complement transcript-only LLM prediction, but that this complementarity is conditional rather than universal. Combining the two predictions significantly improves pairwise ranking accuracy over the transcript-only LLM alone in all evaluated conditions. By contrast, gains in per-participant Pearson r vary across conversation rounds and rating directions, with none significant after correction. Retrospectively, these r gains are concentrated among participants for whom the speech predictor is more accurate. Speech can therefore retain predictive value even when an LLM predicts attraction from transcripts. The relevant question is not simply whether speech helps, but where its complementarity emerges.