Generating Synthetic Doctor-Patient Conversations for Long-form Audio Summarization
Not on the map. Why?
Abstract (arXiv)
Long-context audio reasoning is underserved in both training data and evaluation. Existing benchmarks target short-context tasks, and the open-ended generation tasks most relevant to long-context reasoning pose well-known challenges for automatic evaluation. We propose a synthetic data generation pipeline designed to serve both as a training resource and as a controlled evaluation environment, and instantiate it for first-visit doctor-patient conversations with SOAP note generation as the task. The pipeline has three stages, persona-driven dialogue generation, multi-speaker audio synthesis with overlap/pause modeling, room acoustics, and sound events, and LLM-based reference SOAP note production, built entirely on open-weight models. We release 8,800 synthetic conversations with 1.3k hours of corresponding audio and reference notes. Evaluating current open-weight systems, we find that cascaded approaches still substantially outperform end-to-end models.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 34 of its 38 references that are in the map. First posted on arXiv 7 Apr 2026.
Builds on
34 of 38 references.
References · 34
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2025Gemma 3 Technical Reportcited by 1,824
- 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
- 2025Kimi K2: Open Agentic Intelligencecited by 567
- 2025Qwen3-Omni Technical Reportcited by 497
- 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
- 2019ScispaCy: Fast and Robust Models for Biomedical Natural Language Processingcited by 244
- 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 219
22 more
- 2025Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilitiescited by 133
- 2017Pyroomacoustics: A Python package for audio room simulations and array processing algorithmscited by 132
- 2023ACI-BENCH: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generationcited by 118
- 2026Qwen3-ASR Technical Reportcited by 118
- 2025Baichuan-Omni-1.5 Technical Reportcited by 111
- 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 83
- 2025MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic Evaluation of Audio General Intelligencecited by 61
- 2023NoteChat: A Dataset of Synthetic Doctor-Patient Conversations Conditioned on Clinical Notescited by 48
- 2021Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversationscited by 42
- 2022PriMock57: A Dataset Of Primary Care Mock Consultationscited by 31
- 2025Do Audio LLMs Really LISTEN, or Just Transcribe? Measuring Lexical vs. Acoustic Emotion Cues Reliancecited by 24
- 2021Speech Summarization using Restricted Self-Attentioncited by 23
- 2022Revisiting text decomposition methods for NLI-based factuality scoring of summariescited by 14
- 2025BLAB: Brutally Long Audio Benchcited by 11
- 2021Understanding Medical Conversations: Rich Transcription, Confidence Scores & Information Extractioncited by 10
- 2024Generating Data with Text-to-Speech and Large-Language Models for Conversational Speech Recognitioncited by 9
- 2026Closing the Modality Reasoning Gap for Speech Large Language Modelscited by 9
- 2020The Medical Scribe: Corpus Development and Model Performance Analysescited by 6
- 2026The Cascade Equivalence Hypothesis: When Do Speech LLMs Behave Like ASRrightarrowLLM Pipelines?cited by 5
- 2025Uni-VERSA: Versatile Speech Assessment with a Unified Networkcited by 3
- 2025SDialog: A Python Toolkit for End-to-End Agent Building, User Simulation, Dialog Generation, and Evaluationcited by 3
- 2026Benchmarking Speech Systems for Frontline Health Conversations: The DISPLACE-M Challengecited by 1
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Robust Summarization of Doctor-Patient Conversations: TalTech Systems for the Beyond Transcription Challengecited by 0
- 2026Speech-to-SOAP: End-to-End Summarization of Medical Dialogues: KIT@BeTraC 2026cited by 0
- 2026Bridging the Modality Gap in Long-Form Clinical Audio: A Comparative Study of Lightweight and Heavyweight End-to-End SOAP Generationcited by 0
- 2026CARES: A Controlled Synthetic Benchmark of Speaker Reactions to Soundcited by 0
- 2026CARES: A Controlled Synthetic Benchmark of Speaker Reactions to Sound7 Oct 2026
- 2026Bridging the Modality Gap in Long-Form Clinical Audio: A Comparative Study of Lightweight and Heavyweight End-to-End SOAP Generation13 Sep 2026
- 2026Speech-to-SOAP: End-to-End Summarization of Medical Dialogues: KIT@BeTraC 202625 Aug 2026
- 2026Robust Summarization of Doctor-Patient Conversations: TalTech Systems for the Beyond Transcription Challenge19 Jul 2026