mlsanityData to 7 Oct 2026

Effects of Collaboration on the Performance of Interactive Theme Discovery Systems

arXiv 2408.09030 ↗First posted 16 Aug 2024cs.CL cs.HCcited by 0

Not on the map. Why?

Abstract (arXiv)

NLP-assisted solutions to support qualitative data analysis have gained considerable traction. However, no unified evaluation framework exists which can account for the many different settings in which qualitative researchers may employ them. In this paper, we propose a framework to evaluate the way collaboration settings may produce different research outcomes across a variety of interactive systems. Specifically, we study the impact of synchronous vs. asynchronous collaboration using three different NLP-assisted qualitative research tools and present a comprehensive analysis of the differences in the consistency, cohesiveness, and correctness of their outcomes.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 39 references that are in the map. First posted on arXiv 16 Aug 2024.

Builds on

15 of 39 references.

References · 15

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  3. 2023Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Codingcited by 82
  4. 2021Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherencecited by 74
  5. 2023LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysiscited by 70
  6. 2023LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Codingcited by 64
  7. 2021Modeling Framing in Immigration Discourse on Social Mediacited by 37
  8. 2023CollabCoder: A Lower-barrier, Rigorous Workflow for Inductive Collaborative Qualitative Analysis with Large Language Modelscited by 34
  9. 2021Identifying Morality Frames in Political Tweets using Relational Learningcited by 26
  10. 2023CoAIcoder: Examining the Effectiveness of AI-assisted Human-to-Human Collaboration in Qualitative Analysiscited by 25
  11. 2022A Holistic Framework for Analyzing the COVID-19 Vaccine Debatecited by 18
  12. 2023Interactive Concept Learning for Uncovering Latent Themes in Large Text Collectionscited by 15
3 more
  1. 2021A Query-Driven Topic Modelcited by 5
  2. 2023A User-Centered, Interactive, Human-in-the-Loop Topic Modelling Systemcited by 4
  3. 2023Analysis of Climate Campaigns on Social Media using Bayesian Model Averagingcited by 3

Built on by

Not cited yet.