# SQUINKY! A Corpus of Sentence-level Formality, Informativeness, and Implicature

arXiv: https://arxiv.org/abs/1506.02306 · HTML: https://mlsanity.com/paper/1506.02306/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2015.

## Built on by (most cited first, 11 of 11)

- [Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions](https://mlsanity.com/paper/2306.04140/) (2023)
- [PUB: A Pragmatics Understanding Benchmark for Assessing LLMs' Pragmatics Capabilities](https://mlsanity.com/paper/2401.07078/) (2024)
- [XFORMAL: A Benchmark for Multilingual Formality Style Transfer](https://mlsanity.com/paper/2104.04108/) (2021)
- [Oolong: Evaluating Long Context Reasoning and Aggregation Capabilities](https://mlsanity.com/paper/2511.02817/) (2025)
- [Evaluating the Evaluation Metrics for Style Transfer: A Case Study in Multilingual Formality Transfer](https://mlsanity.com/paper/2110.10668/) (2021)
- [ALDi: Quantifying the Arabic Level of Dialectness of Text](https://mlsanity.com/paper/2310.13747/) (2023)
- [TyDiP: A Dataset for Politeness Classification in Nine Typologically Diverse Languages](https://mlsanity.com/paper/2211.16496/) (2022)
- [Understand the Implication: Learning to Think for Pragmatic Understanding](https://mlsanity.com/paper/2506.13559/) (2025)
- [In What Languages are Generative Language Models the Most Formal? Analyzing Formality Distribution across Languages](https://mlsanity.com/paper/2302.12299/) (2023)
- [An Attention-Based Model for Predicting Contextual Informativeness and Curriculum Learning Applications](https://mlsanity.com/paper/2204.09885/) (2022)
- [Beyond Discrete Genres: Mapping News Items onto a Multidimensional Framework of Genre Cues](https://mlsanity.com/paper/2212.04185/) (2022)

## Abstract (arXiv)

We introduce a corpus of 7,032 sentences rated by human annotators for formality, informativeness, and implicature on a 1-7 scale. The corpus was annotated using Amazon Mechanical Turk. Reliability in the obtained judgments was examined by comparing mean ratings across two MTurk experiments, and correlation with pilot annotations (on sentence formality) conducted in a more controlled setting. Despite the subjectivity and inherent difficulty of the annotation task, correlations between mean ratings were quite encouraging, especially on formality and informativeness. We further explored correlation between the three linguistic variables, genre-wise variation of ratings and correlations within genres, compatibility with automatic stylistic scoring, and sentential make-up of a document in terms of style. To date, our corpus is the largest sentence-level annotated corpus released for formality, informativeness, and implicature.