# Classifying Constructive Comments

arXiv: https://arxiv.org/abs/2004.05476 · HTML: https://mlsanity.com/paper/2004.05476/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2020.

## Built on by (most cited first, 8 of 8)

- [Conversations Gone Alright: Quantifying and Predicting Prosocial Outcomes in Online Conversations](https://mlsanity.com/paper/2102.08368/) (2021)
- [Constructive and Toxic Speech Detection for Open-domain Social Media Comments in Vietnamese](https://mlsanity.com/paper/2103.10069/) (2021)
- [Towards Agile Text Classifiers for Everyone](https://mlsanity.com/paper/2302.06541/) (2023)
- [Towards Equal Gender Representation in the Annotations of Toxic Language Detection](https://mlsanity.com/paper/2106.02183/) (2021)
- [AI and the Future of Digital Public Squares](https://mlsanity.com/paper/2412.09988/) (2024)
- [Representative Ranking for Deliberation in the Public Sphere](https://mlsanity.com/paper/2503.18962/) (2025)
- [Hybrid moderation in the newsroom: Recommending featured posts to content moderators](https://mlsanity.com/paper/2307.07317/) (2023)
- [The Impact of Featuring Comments in Online Discussions](https://mlsanity.com/paper/2412.02369/) (2024)

## Abstract (arXiv)

We introduce the Constructive Comments Corpus (C3), comprised of 12,000 annotated news comments, intended to help build new tools for online communities to improve the quality of their discussions. We define constructive comments as high-quality comments that make a contribution to the conversation. We explain the crowd worker annotation scheme and define a taxonomy of sub-characteristics of constructiveness. The quality of the annotation scheme and the resulting dataset is evaluated using measurements of inter-annotator agreement, expert assessment of a sample, and by the constructiveness sub-characteristics, which we show provide a proxy for the general constructiveness concept. We provide models for constructiveness trained on C3 using both feature-based and a variety of deep learning approaches and demonstrate that these models capture general rather than topic- or domain-specific characteristics of constructiveness, through domain adaptation experiments. We examine the role that length plays in our models, as comment length could be easily gamed if models depend heavily upon this feature. By examining the errors made by each model and their distribution by length, we show that the best performing models are less correlated with comment length.The constructiveness corpus and our experiments pave the way for a moderation tool focused on promoting comments that make a contribution, rather than only filtering out undesirable content.