# Thematic Fit Bits: Annotation Quality and Quantity Interplay for Event Participant Representation

arXiv: https://arxiv.org/abs/2105.06097 · HTML: https://mlsanity.com/paper/2105.06097/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 44 references that are in the map.
- First posted on arXiv 13 May 2021.

## Builds on (most cited first, 4 of 4)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling](https://mlsanity.com/paper/1805.04787/) (2018)
- [Measuring Thematic Fit with Distributional Feature Overlap](https://mlsanity.com/paper/1707.05967/) (2017)

## Built on by (most cited first, 3 of 3)

- [Automotive Perception Software Development: An Empirical Investigation into Data, Annotation, and Ecosystem Challenges](https://mlsanity.com/paper/2303.05947/) (2023)
- [Where's the Learning in Representation Learning for Compositional Semantics and the Case of Thematic Fit](https://mlsanity.com/paper/2208.04749/) (2022)
- [Uncovering Autoregressive LLM Knowledge of Thematic Fit in Event Representation](https://mlsanity.com/paper/2410.15173/) (2024)

## Abstract (arXiv)

Modeling thematic fit (a verb--argument compositional semantics task) currently requires a very large burden of labeled data. We take a linguistically machine-annotated large corpus and replace corpus layers with output from higher-quality, more modern taggers. We compare the old and new corpus versions' impact on a verb--argument fit modeling task, using a high-performing neural approach. We discover that higher annotation quality dramatically reduces our data requirement while demonstrating better supervised predicate-argument classification. But in applying the model to psycholinguistic tasks outside the training objective, we see clear gains at scale, but only in one of two thematic fit estimation tasks, and no clear gains on the other. We also see that quality improves with training size, but perhaps plateauing or even declining in one task. Last, we tested the effect of role set size. All this suggests that the quality/quantity interplay is not all you need. We replicate previous studies while modifying certain role representation details and set a new state-of-the-art in event modeling, using a fraction of the data. We make the new corpus version public.