# Gamified Crowdsourcing for Idiom Corpora Construction

arXiv: https://arxiv.org/abs/2102.00881 · HTML: https://mlsanity.com/paper/2102.00881/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 54 references that are in the map.
- First posted on arXiv 1 Feb 2021.

## Builds on (most cited first, 4 of 4)

- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Italy goes to Stanford: a collection of CoreNLP modules for Italian](https://mlsanity.com/paper/1609.06204/) (2016)
- [How to Work a Crowd: Developing Crowd Capital Through Crowdsourcing](https://mlsanity.com/paper/1702.04214/) (2017)

## Built on by (most cited first, 3 of 3)

- [Lessons Learned from a Citizen Science Project for Natural Language Processing](https://mlsanity.com/paper/2304.12836/) (2023)
- [A Parallel Cross-Lingual Benchmark for Multimodal Idiomaticity Understanding](https://mlsanity.com/paper/2601.08645/) (2026)
- [IdioLink: Retrieving Meaning Beyond Words Across Idiomatic and Literal Expressions](https://mlsanity.com/paper/2605.22247/) (2026)

## Abstract (arXiv)

Learning idiomatic expressions is seen as one of the most challenging stages in second language learning because of their unpredictable meaning. A similar situation holds for their identification within natural language processing applications such as machine translation and parsing. The lack of high-quality usage samples exacerbates this challenge not only for humans but also for artificial intelligence systems. This article introduces a gamified crowdsourcing approach for collecting language learning materials for idiomatic expressions; a messaging bot is designed as an asynchronous multiplayer game for native speakers who compete with each other while providing idiomatic and nonidiomatic usage examples and rating other players' entries. As opposed to classical crowdprocessing annotation efforts in the field, for the first time in the literature, a crowdcreating & crowdrating approach is implemented and tested for idiom corpora construction. The approach is language independent and evaluated on two languages in comparison to traditional data preparation techniques in the field. The reaction of the crowd is monitored under different motivational means (namely, gamification affordances and monetary rewards). The results reveal that the proposed approach is powerful in collecting the targeted materials, and although being an explicit crowdsourcing approach, it is found entertaining and useful by the crowd. The approach has been shown to have the potential to speed up the construction of idiom corpora for different natural languages to be used as second language learning material, training data for supervised idiom identification systems, or samples for lexicographic studies.