# Topic-Conversation Relevance (TCR) Dataset and Benchmarks

arXiv: https://arxiv.org/abs/2411.00038 · HTML: https://mlsanity.com/paper/2411.00038/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 34 references that are in the map.
- First posted on arXiv 29 Oct 2024.

## Builds on (most cited first, 12 of 12)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization](https://mlsanity.com/paper/2104.05938/) (2021)
- [Understanding Social Reasoning in Language Models with Language Models](https://mlsanity.com/paper/2306.15448/) (2023)
- [MeetingBank: A Benchmark Dataset for Meeting Summarization](https://mlsanity.com/paper/2305.17529/) (2023)
- [Can language models handle recursively nested grammatical structures? A case study on comparing models and humans](https://mlsanity.com/paper/2210.15303/) (2022)
- [Do large language models resemble humans in language use?](https://mlsanity.com/paper/2303.08014/) (2023)
- [Meeting Effectiveness and Inclusiveness in Remote Collaboration](https://mlsanity.com/paper/2102.09803/) (2021)
- [ALIGNMEET: A Comprehensive Tool for Meeting Annotation, Alignment, and Evaluation](https://mlsanity.com/paper/2205.05433/) (2022)
- [Team Hitachi @ AutoMin 2021: Reference-free Automatic Minuting Pipeline with Argument Structure Construction over Topic-based Summarization](https://mlsanity.com/paper/2112.02741/) (2021)
- [Improving Meeting Inclusiveness using Speech Interruption Analysis](https://mlsanity.com/paper/2304.00658/) (2023)
- [Mental Models of Meeting Goals: Supporting Intentionality in Meeting Technologies](https://mlsanity.com/paper/2402.18526/) (2024)

## Built on by (most cited first, 5 of 5)

- [Are LLM Agents Behaviorally Coherent? Latent Profiles for Social Simulation](https://mlsanity.com/paper/2509.03736/) (2025)
- [Evaluating LLM-Simulated Conversations in Modeling Inconsistent and Uncollaborative Behaviors in Human Social Interaction](https://mlsanity.com/paper/2603.17094/) (2026)
- [CIG: Measuring Conversational Information Gain in Deliberative Dialogues with Semantic Memory Dynamics](https://mlsanity.com/paper/2604.15647/) (2026)
- [GLARE: Generative Learning via Adversarial Reward Estimation For Social Dynamics Forecasting](https://mlsanity.com/paper/2609.12165/) (2026)
- [Zero-Shot Cue-Grounded Topic Segmentation of Spoken Documents](https://mlsanity.com/paper/2609.34425/) (2026)

## Abstract (arXiv)

Workplace meetings are vital to organizational collaboration, yet a large percentage of meetings are rated as ineffective. To help improve meeting effectiveness by understanding if the conversation is on topic, we create a comprehensive Topic-Conversation Relevance (TCR) dataset that covers a variety of domains and meeting styles. The TCR dataset includes 1,500 unique meetings, 22 million words in transcripts, and over 15,000 meeting topics, sourced from both newly collected Speech Interruption Meeting (SIM) data and existing public datasets. Along with the text data, we also open source scripts to generate synthetic meetings or create augmented meetings from the TCR dataset to enhance data diversity. For each data source, benchmarks are created using GPT-4 to evaluate the model accuracy in understanding transcription-topic relevance.