# Gradient Imitation Reinforcement Learning for Low Resource Relation Extraction

arXiv: https://arxiv.org/abs/2109.06415 · HTML: https://mlsanity.com/paper/2109.06415/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 14 Sep 2021.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019)
- [Deep Reinforcement Learning for Dialogue Generation](https://mlsanity.com/paper/1606.01541/) (2016)
- [Matching the Blanks: Distributional Similarity for Relation Learning](https://mlsanity.com/paper/1906.03158/) (2019)
- [Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learning](https://mlsanity.com/paper/2010.15875/) (2020)
- [Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations](https://mlsanity.com/paper/1805.06201/) (2018)
- [Data Augmentation using Pre-trained Transformer Models](https://mlsanity.com/paper/2003.02245/) (2020)
- [Adapting Auxiliary Losses Using Gradient Similarity](https://mlsanity.com/paper/1812.02224/) (2018)
- [Reasoning with Latent Structure Refinement for Document-Level Relation Extraction](https://mlsanity.com/paper/2005.06312/) (2020)
- [An Analysis of Simple Data Augmentation for Named Entity Recognition](https://mlsanity.com/paper/2010.11683/) (2020)
- [SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals](https://mlsanity.com/paper/1911.10422/) (2019)
- [A Hierarchical Framework for Relation Extraction with Reinforcement Learning](https://mlsanity.com/paper/1811.03925/) (2018)
- [Soft Contextual Data Augmentation for Neural Machine Translation](https://mlsanity.com/paper/1905.10523/) (2019)
- [SelfORE: Self-supervised Relational Feature Learning for Open Relation Extraction](https://mlsanity.com/paper/2004.02438/) (2020)
- [Semi-supervised Relation Extraction via Incremental Meta Self-Training](https://mlsanity.com/paper/2010.16410/) (2020)
- [On-line Active Reward Learning for Policy Optimisation in Spoken Dialogue Systems](https://mlsanity.com/paper/1605.07669/) (2016)
- [Noisy-Labeled NER with Confidence Estimation](https://mlsanity.com/paper/2104.04318/) (2021)
- [Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning](https://mlsanity.com/paper/1603.07954/) (2016)
- [Learning Dual Retrieval Module for Semi-supervised Relation Extraction](https://mlsanity.com/paper/1902.07814/) (2019)
- [Incorporating Relation Paths in Neural Relation Extraction](https://mlsanity.com/paper/1609.07479/) (2016)
- [Exploit Multiple Reference Graphs for Semi-supervised Relation Extraction](https://mlsanity.com/paper/2010.11383/) (2020)

## Built on by (most cited first, 30 of 33)

- [HiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extraction](https://mlsanity.com/paper/2205.02225/) (2022)
- [Character-level White-Box Adversarial Attacks against Transformers via Attachable Subwords Substitution](https://mlsanity.com/paper/2210.17004/) (2022)
- [Semantic Enhanced Text-to-SQL Parsing via Iteratively Learning Schema Linking Graph](https://mlsanity.com/paper/2208.03903/) (2022)
- [Unsupervised Entity Alignment for Temporal Knowledge Graphs](https://mlsanity.com/paper/2302.00796/) (2023)
- [Read it Twice: Towards Faithfully Interpretable Fact Verification by Revisiting Evidence](https://mlsanity.com/paper/2305.03507/) (2023)
- [PSP: Pre-trained Soft Prompts for Few-Shot Abstractive Summarization](https://mlsanity.com/paper/2204.04413/) (2022)
- [Pair-Level Supervised Contrastive Learning for Natural Language Inference](https://mlsanity.com/paper/2201.10927/) (2022)
- [Domain-Specific NER via Retrieving Correlated Samples](https://mlsanity.com/paper/2208.12995/) (2022)
- [Information Extraction in Low-Resource Scenarios: Survey and Perspective](https://mlsanity.com/paper/2202.08063/) (2022)
- [GDA: Generative Data Augmentation Techniques for Relation Extraction Tasks](https://mlsanity.com/paper/2305.16663/) (2023)
- [RAPL: A Relation-Aware Prototype Learning Approach for Few-Shot Document-Level Relation Extraction](https://mlsanity.com/paper/2310.15743/) (2023)
- [Deep Structural Knowledge Exploitation and Synergy for Estimating Node Importance Value on Heterogeneous Information Networks](https://mlsanity.com/paper/2402.12411/) (2024)
- [A Multi-level Supervised Contrastive Learning Framework for Low-Resource Natural Language Inference](https://mlsanity.com/paper/2205.15550/) (2022)
- [Uncertainty-Aware Bootstrap Learning for Joint Extraction on Distantly-Supervised Data](https://mlsanity.com/paper/2305.03827/) (2023)
- [Multimodal Relation Extraction with Cross-Modal Retrieval and Synthesis](https://mlsanity.com/paper/2305.16166/) (2023)
- [Graph Component Contrastive Learning for Concept Relatedness Estimation](https://mlsanity.com/paper/2206.12556/) (2022)
- [Entity-to-Text based Data Augmentation for various Named Entity Recognition Tasks](https://mlsanity.com/paper/2210.10343/) (2022)
- [Think Rationally about What You See: Continuous Rationale Extraction for Relation Extraction](https://mlsanity.com/paper/2305.03503/) (2023)
- [Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study](https://mlsanity.com/paper/2210.10678/) (2022)
- [Query-based Instance Discrimination Network for Relational Triple Extraction](https://mlsanity.com/paper/2211.01797/) (2022)
- [Prompt Me Up: Unleashing the Power of Alignments for Multimodal Entity and Relation Extraction](https://mlsanity.com/paper/2310.16822/) (2023)
- [Give Me More Details: Improving Fact-Checking with Latent Retrieval](https://mlsanity.com/paper/2305.16128/) (2023)
- [Targeted Augmentation for Low-Resource Event Extraction](https://mlsanity.com/paper/2405.08729/) (2024)
- [Improve Event Extraction via Self-Training with Gradient Guidance](https://mlsanity.com/paper/2205.12490/) (2022)
- [Gaussian Prior Reinforcement Learning for Nested Named Entity Recognition](https://mlsanity.com/paper/2305.07266/) (2023)
- [Scene Graph Modification as Incremental Structure Expanding](https://mlsanity.com/paper/2209.09093/) (2022)
- [Extending Phrase Grounding with Pronouns in Visual Dialogues](https://mlsanity.com/paper/2210.12658/) (2022)
- [Jointprop: Joint Semi-supervised Learning for Entity and Relation Extraction with Heterogeneous Graph-based Propagation](https://mlsanity.com/paper/2305.15872/) (2023)
- [Distantly-Supervised Joint Extraction with Noise-Robust Learning](https://mlsanity.com/paper/2310.04994/) (2023)
- [Semi-supervised Relation Extraction via Data Augmentation and Consistency-training](https://mlsanity.com/paper/2306.10153/) (2023)

## Abstract (arXiv)

Low-resource Relation Extraction (LRE) aims to extract relation facts from limited labeled corpora when human annotation is scarce. Existing works either utilize self-training scheme to generate pseudo labels that will cause the gradual drift problem, or leverage meta-learning scheme which does not solicit feedback explicitly. To alleviate selection bias due to the lack of feedback loops in existing LRE learning paradigms, we developed a Gradient Imitation Reinforcement Learning method to encourage pseudo label data to imitate the gradient descent direction on labeled data and bootstrap its optimization capability through trial and error. We also propose a framework called GradLRE, which handles two major scenarios in low-resource relation extraction. Besides the scenario where unlabeled data is sufficient, GradLRE handles the situation where no unlabeled data is available, by exploiting a contextualized augmentation method to generate data. Experimental results on two public datasets demonstrate the effectiveness of GradLRE on low resource relation extraction when comparing with baselines.