# The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task

arXiv: https://arxiv.org/abs/1702.01841 · HTML: https://mlsanity.com/paper/1702.01841/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Feb 2017.

## Built on by (most cited first, 30 of 39)

- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Symbolic Knowledge Distillation: from General Language Models to Commonsense Models](https://mlsanity.com/paper/2110.07178/) (2021)
- [Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs](https://mlsanity.com/paper/2210.13312/) (2022)
- [Are We Done with MMLU?](https://mlsanity.com/paper/2406.04127/) (2024)
- [Challenges in Automated Debiasing for Toxic Language Detection](https://mlsanity.com/paper/2102.00086/) (2021)
- [Prompting for Multimodal Hateful Meme Classification](https://mlsanity.com/paper/2302.04156/) (2023)
- [Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets](https://mlsanity.com/paper/2203.12942/) (2022)
- [Don't Blame the Annotator: Bias Already Starts in the Annotation Instructions](https://mlsanity.com/paper/2205.00415/) (2022)
- [It's not Rocket Science : Interpreting Figurative Language in Narratives](https://mlsanity.com/paper/2109.00087/) (2021)
- [Perceptual Score: What Data Modalities Does Your Model Perceive?](https://mlsanity.com/paper/2110.14375/) (2021)
- [Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants](https://mlsanity.com/paper/2112.09062/) (2021)
- [Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View](https://mlsanity.com/paper/2203.12258/) (2022)
- [Automatic Story Generation: Challenges and Attempts](https://mlsanity.com/paper/2102.12634/) (2021)
- [It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance](https://mlsanity.com/paper/2305.09022/) (2023)
- [LOT: A Story-Centric Benchmark for Evaluating Chinese Long Text Understanding and Generation](https://mlsanity.com/paper/2108.12960/) (2021)
- [Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning](https://mlsanity.com/paper/2109.04144/) (2021)
- [Tiered Reasoning for Intuitive Physics: Toward Verifiable Commonsense Language Understanding](https://mlsanity.com/paper/2109.04947/) (2021)
- [On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlations](https://mlsanity.com/paper/2204.12708/) (2022)
- [Is Everything in Order? A Simple Way to Order Sentences](https://mlsanity.com/paper/2104.07064/) (2021)
- [Behind the Scenes: An Exploration of Trigger Biases Problem in Few-Shot Event Classification](https://mlsanity.com/paper/2108.12844/) (2021)
- [Beyond the Tip of the Iceberg: Assessing Coherence of Text Classifiers](https://mlsanity.com/paper/2109.04922/) (2021)
- [A Temporal Variational Model for Story Generation](https://mlsanity.com/paper/2109.06807/) (2021)
- [Fighting Bias with Bias: Promoting Model Robustness by Amplifying Dataset Biases](https://mlsanity.com/paper/2305.18917/) (2023)
- [InfoVisDial: An Informative Visual Dialogue Dataset by Bridging Large Multimodal and Language Models](https://mlsanity.com/paper/2312.13503/) (2023)
- [OffensiveLang: A Community Based Implicit Offensive Language Dataset](https://mlsanity.com/paper/2403.02472/) (2024)
- [A Proposal to Study "Is High Quality Data All We Need?"](https://mlsanity.com/paper/2203.06404/) (2022)
- [From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoning](https://mlsanity.com/paper/2310.18364/) (2023)
- [Challenges in Procedural Multimodal Machine Comprehension:A Novel Way To Benchmark](https://mlsanity.com/paper/2110.11899/) (2021)
- [A Survey of Parameters Associated with the Quality of Benchmarks in NLP](https://mlsanity.com/paper/2210.07566/) (2022)

## Abstract (arXiv)

A writer's style depends not just on personal traits but also on her intent and mental state. In this paper, we show how variants of the same writing task can lead to measurable differences in writing style. We present a case study based on the story cloze task (Mostafazadeh et al., 2016a), where annotators were assigned similar writing tasks with different constraints: (1) writing an entire story, (2) adding a story ending for a given story context, and (3) adding an incoherent ending to a story. We show that a simple linear classifier informed by stylistic features is able to successfully distinguish among the three cases, without even looking at the story context. In addition, combining our stylistic features with language model predictions reaches state of the art performance on the story cloze challenge. Our results demonstrate that different task framings can dramatically affect the way people write.