mlsanityData to 13 Oct 2026

Speech Signals Complement LLMs for Predicting Interpersonal Attraction in Speed Dating

arXiv 2607.23037 ↗First posted 25 Jul 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) can predict interpersonal attraction from conversation transcripts, but it remains unclear what a speech predictor can add beyond transcript-only LLM prediction. Using Japanese speed-dating conversations, we combine predictions from a transcript-only LLM and a supervised speech predictor to estimate participants' reported liking of their partners. We show that speech can complement transcript-only LLM prediction, but that this complementarity is conditional rather than universal. Combining the two predictions significantly improves pairwise ranking accuracy over the transcript-only LLM alone in all evaluated conditions. By contrast, gains in per-participant Pearson r vary across conversation rounds and rating directions, with none significant after correction. Retrospectively, these r gains are concentrated among participants for whom the speech predictor is more accurate. Speech can therefore retain predictive value even when an LLM predicts attraction from transcripts. The relevant question is not simply whether speech helps, but where its complementarity emerges.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 9 of its 41 references that are in the map. First posted on arXiv 25 Jul 2026.

Builds on

9 of 41 references.

References · 9

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,870
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,313
  4. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 464
  5. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  6. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 284
  7. 2025When Large Language Models are Reliable for Judging Empathic Communicationcited by 21
  8. 2025Using LLMs for Late Multimodal Sensor Fusion for Activity Recognitioncited by 5
  9. 2024Multimodal Fusion with LLMs for Engagement Prediction in Natural Conversationcited by 3

Built on by

Not cited yet.