# Pseudo Zero Pronoun Resolution Improves Zero Anaphora Resolution

arXiv: https://arxiv.org/abs/2104.07425 · HTML: https://mlsanity.com/paper/2104.07425/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 42 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [BERT for Coreference Resolution: Baselines and Analysis](https://mlsanity.com/paper/1908.09091/) (2019)
- [Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification](https://mlsanity.com/paper/2010.13641/) (2020)
- [A Surprisingly Robust Trick for Winograd Schema Challenge](https://mlsanity.com/paper/1905.06290/) (2019)
- [An Empirical Study of Contextual Data Augmentation for Japanese Zero Anaphora Resolution](https://mlsanity.com/paper/2011.00948/) (2020)
- [Generating and Exploiting Large-scale Pseudo Training Data for Zero Pronoun Resolution](https://mlsanity.com/paper/1606.01603/) (2016)
- [Multi-task Learning for Japanese Predicate Argument Structure Analysis](https://mlsanity.com/paper/1904.02244/) (2019)
- [Distance-Free Modeling of Multi-Predicate Interactions in End-to-End Japanese Predicate-Argument Structure Analysis](https://mlsanity.com/paper/1806.03869/) (2018)
- [Neural Adversarial Training for Semi-supervised Japanese Predicate-argument Structure Analysis](https://mlsanity.com/paper/1806.00971/) (2018)
- [Semantic Role Labeling with Associated Memory Network](https://mlsanity.com/paper/1908.02367/) (2019)

## Built on by (most cited first, 4 of 4)

- [Do language models make human-like predictions about the coreferents of Italian anaphoric zero pronouns?](https://mlsanity.com/paper/2208.14554/) (2022)
- [Joint Coreference Resolution for Zeros and non-Zeros in Arabic](https://mlsanity.com/paper/2210.12169/) (2022)
- [To Drop or Not to Drop? Predicting Argument Ellipsis Judgments: A Case Study in Japanese](https://mlsanity.com/paper/2404.11315/) (2024)
- [How Much Do LLMs Know About Chinese Zero Pronouns?](https://mlsanity.com/paper/2605.31056/) (2026)

## Abstract (arXiv)

Masked language models (MLMs) have contributed to drastic performance improvements with regard to zero anaphora resolution (ZAR). To further improve this approach, in this study, we made two proposals. The first is a new pretraining task that trains MLMs on anaphoric relations with explicit supervision, and the second proposal is a new finetuning method that remedies a notorious issue, the pretrain-finetune discrepancy. Our experiments on Japanese ZAR demonstrated that our two proposals boost the state-of-the-art performance, and our detailed analysis provides new insights on the remaining challenges.