# TYPIC: A Corpus of Template-Based Diagnostic Comments on Argumentation

arXiv: https://arxiv.org/abs/2201.06674 · HTML: https://mlsanity.com/paper/2201.06674/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 47 references that are in the map.
- First posted on arXiv 18 Jan 2022.

## Builds on (most cited first, 6 of 6)

- [How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation](https://mlsanity.com/paper/1603.08023/) (2016)
- [To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation](https://mlsanity.com/paper/2107.10821/) (2021)
- [Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics](https://mlsanity.com/paper/2006.06264/) (2020)
- [MultiVerS: Improving scientific claim verification with weak supervision and full-document context](https://mlsanity.com/paper/2112.01640/) (2021)
- [ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning](https://mlsanity.com/paper/2104.07644/) (2021)
- [Effective Interfaces for Student-Driven Revision Sessions for Argumentative Writing](https://mlsanity.com/paper/2107.07018/) (2021)

## Built on by (most cited first, 3 of 3)

- [Towards a Perspectivist Turn in Argument Quality Assessment](https://mlsanity.com/paper/2502.14501/) (2025)
- [LPAttack: A Feasible Annotation Scheme for Capturing Logic Pattern of Attacks in Arguments](https://mlsanity.com/paper/2204.01512/) (2022)
- [Teach Me How to Improve My Argumentation Skills: A Survey on Feedback in Argumentation](https://mlsanity.com/paper/2307.15341/) (2023)

## Abstract (arXiv)

Providing feedback on the argumentation of the learner is essential for developing critical thinking skills, however, it requires a lot of time and effort. To mitigate the overload on teachers, we aim to automate a process of providing feedback, especially giving diagnostic comments which point out the weaknesses inherent in the argumentation. It is recommended to give specific diagnostic comments so that learners can recognize the diagnosis without misinterpretation. However, it is not obvious how the task of providing specific diagnostic comments should be formulated. We present a formulation of the task as template selection and slot filling to make an automatic evaluation easier and the behavior of the model more tractable. The key to the formulation is the possibility of creating a template set that is sufficient for practical use. In this paper, we define three criteria that a template set should satisfy: expressiveness, informativeness, and uniqueness, and verify the feasibility of creating a template set that satisfies these criteria as a first trial. We will show that it is feasible through an annotation study that converts diagnostic comments given in a text to a template format. The corpus used in the annotation study is publicly available.