# Where's the Learning in Representation Learning for Compositional Semantics and the Case of Thematic Fit

arXiv: https://arxiv.org/abs/2208.04749 · HTML: https://mlsanity.com/paper/2208.04749/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 25 references that are in the map.
- First posted on arXiv 9 Aug 2022.

## Builds on (most cited first, 4 of 4)

- [Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling](https://mlsanity.com/paper/1805.04787/) (2018)
- [A comparative evaluation and analysis of three generations of Distributional Semantic Models](https://mlsanity.com/paper/2105.09825/) (2021)
- [What do Toothbrushes do in the Kitchen? How Transformers Think our World is Structured](https://mlsanity.com/paper/2204.05673/) (2022)
- [Thematic Fit Bits: Annotation Quality and Quantity Interplay for Event Participant Representation](https://mlsanity.com/paper/2105.06097/) (2021)

## Built on by (most cited first, 2 of 2)

- [Structural Ambiguity and its Disambiguation in Language Model Based Parsers: the Case of Dutch Clause Relativization](https://mlsanity.com/paper/2305.14917/) (2023)
- [Uncovering Autoregressive LLM Knowledge of Thematic Fit in Event Representation](https://mlsanity.com/paper/2410.15173/) (2024)

## Abstract (arXiv)

Observing that for certain NLP tasks, such as semantic role prediction or thematic fit estimation, random embeddings perform as well as pretrained embeddings, we explore what settings allow for this and examine where most of the learning is encoded: the word embeddings, the semantic role embeddings, or ``the network''. We find nuanced answers, depending on the task and its relation to the training objective. We examine these representation learning aspects in multi-task learning, where role prediction and role-filling are supervised tasks, while several thematic fit tasks are outside the models' direct supervision. We observe a non-monotonous relation between some tasks' quality score and the training data size. In order to better understand this observation, we analyze these results using easier, per-verb versions of these tasks.