# Improve Event Extraction via Self-Training with Gradient Guidance

arXiv: https://arxiv.org/abs/2205.12490 · HTML: https://mlsanity.com/paper/2205.12490/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 62 references that are in the map.
- First posted on arXiv 25 May 2022.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Meta Pseudo Labels](https://mlsanity.com/paper/2003.10580/) (2020)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019)
- [Unified Structure Generation for Universal Information Extraction](https://mlsanity.com/paper/2203.12277/) (2022)
- [Event Extraction by Answering (Almost) Natural Questions](https://mlsanity.com/paper/2004.13625/) (2020)
- [Improved Noisy Student Training for Automatic Speech Recognition](https://mlsanity.com/paper/2005.09629/) (2020)
- [Self-Training for End-to-End Speech Recognition](https://mlsanity.com/paper/1909.09116/) (2019)
- [Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction](https://mlsanity.com/paper/2106.09232/) (2021)
- [Self-training Improves Pre-training for Natural Language Understanding](https://mlsanity.com/paper/2010.02194/) (2020)
- [DEGREE: A Data-Efficient Generation-Based Event Extraction Model](https://mlsanity.com/paper/2108.12724/) (2021)
- [Zero-Shot Transfer Learning for Event Extraction](https://mlsanity.com/paper/1707.01066/) (2017)
- [Transition-based Parsing with Stack-Transformers](https://mlsanity.com/paper/2010.10669/) (2020)
- [CLEVE: Contrastive Pre-training for Event Extraction](https://mlsanity.com/paper/2105.14485/) (2021)
- [Semi-supervised Relation Extraction via Incremental Meta Self-Training](https://mlsanity.com/paper/2010.16410/) (2020)
- [Gradient Imitation Reinforcement Learning for Low Resource Relation Extraction](https://mlsanity.com/paper/2109.06415/) (2021)
- [Noisy-Labeled NER with Confidence Estimation](https://mlsanity.com/paper/2104.04318/) (2021)
- [Query and Extract: Refining Event Extraction as Type-oriented Binary Decoding](https://mlsanity.com/paper/2110.07476/) (2021)
- [OntoED: Low-resource Event Detection with Ontology Embedding](https://mlsanity.com/paper/2105.10922/) (2021)
- [Cross-Task Instance Representation Interactions and Label Dependencies for Joint Information Extraction with Graph Convolutional Networks](https://mlsanity.com/paper/2103.09330/) (2021)
- [Semi-Supervised Event Extraction with Paraphrase Clusters](https://mlsanity.com/paper/1808.08622/) (2018)
- [Towards Semi-Supervised Learning for Deep Semantic Role Labeling](https://mlsanity.com/paper/1808.09543/) (2018)
- [Improved Latent Tree Induction with Distant Supervision via Span Constraints](https://mlsanity.com/paper/2109.05112/) (2021)

## Built on by (most cited first, 3 of 3)

- [CMNEE: A Large-Scale Document-Level Event Extraction Dataset based on Open-Source Chinese Military News](https://mlsanity.com/paper/2404.12242/) (2024)
- [Harvesting Events from Multiple Sources: Towards a Cross-Document Event Extraction Paradigm](https://mlsanity.com/paper/2406.16021/) (2024)
- [Error-driven Data-efficient Large Multimodal Model Tuning](https://mlsanity.com/paper/2412.15652/) (2024)

## Abstract (arXiv)

Data scarcity has been the main factor that hinders the progress of event extraction. To overcome this issue, we propose a Self-Training with Feedback (STF) framework that leverages the large-scale unlabeled data and acquires feedback for each new event prediction from the unlabeled data by comparing it to the Abstract Meaning Representation (AMR) graph of the same sentence. Specifically, STF consists of (1) a base event extraction model trained on existing event annotations and then applied to large-scale unlabeled corpora to predict new event mentions as pseudo training samples, and (2) a novel scoring model that takes in each new predicted event trigger, an argument, its argument role, as well as their paths in the AMR graph to estimate a compatibility score indicating the correctness of the pseudo label. The compatibility scores further act as feedback to encourage or discourage the model learning on the pseudo labels during self-training. Experimental results on three benchmark datasets, including ACE05-E, ACE05-E+, and ERE, demonstrate the effectiveness of the STF framework on event extraction, especially event argument extraction, with significant performance gain over the base event extraction models and strong baselines. Our experimental analysis further shows that STF is a generic framework as it can be applied to improve most, if not all, event extraction models by leveraging large-scale unlabeled data, even when high-quality AMR graph annotations are not available.