# SHINE: Syntax-augmented Hierarchical Interactive Encoder for Zero-shot Cross-lingual Information Extraction

arXiv: https://arxiv.org/abs/2305.12389 · HTML: https://mlsanity.com/paper/2305.12389/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 92 references that are in the map.
- First posted on arXiv 21 May 2023.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT](https://mlsanity.com/paper/1904.09077/) (2019)
- [SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization](https://mlsanity.com/paper/1911.03437/) (2019)
- [A Unified MRC Framework for Named Entity Recognition](https://mlsanity.com/paper/1910.11476/) (2019)
- [Representing Long-Range Context for Graph Neural Networks with Global Attention](https://mlsanity.com/paper/2201.08821/) (2022)
- [Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!](https://mlsanity.com/paper/2303.08559/) (2023)
- [A General Framework for Information Extraction using Dynamic Span Graphs](https://mlsanity.com/paper/1904.03296/) (2019)
- [Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning](https://mlsanity.com/paper/2105.03654/) (2021)
- [Exploring the Feasibility of ChatGPT for Event Extraction](https://mlsanity.com/paper/2303.03836/) (2023)
- [Jointly Multiple Events Extraction via Attention-based Graph Information Aggregation](https://mlsanity.com/paper/1809.09078/) (2018)
- [Zero-Shot Transfer Learning for Event Extraction](https://mlsanity.com/paper/1707.01066/) (2017)
- [ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora](https://mlsanity.com/paper/2012.15674/) (2020)
- [SpanNER: Named Entity Re-/Recognition as Span Prediction](https://mlsanity.com/paper/2106.00641/) (2021)
- [Neural Cross-Lingual Named Entity Recognition with Minimal Resources](https://mlsanity.com/paper/1808.09861/) (2018)
- [GATE: Graph Attention Transformer Encoder for Cross-lingual Relation and Event Extraction](https://mlsanity.com/paper/2010.03009/) (2020)
- [Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction](https://mlsanity.com/paper/2203.08308/) (2022)
- [Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Language](https://mlsanity.com/paper/2004.12440/) (2020)
- [Adversarial training for multi-context joint entity and relation extraction](https://mlsanity.com/paper/1808.06876/) (2018)
- [Adversarial Learning with Contextual Embeddings for Zero-resource Cross-lingual Classification and NER](https://mlsanity.com/paper/1909.00153/) (2019)
- [UniTrans: Unifying Model Transfer and Data Transfer for Cross-Lingual Named Entity Recognition with Unlabeled Data](https://mlsanity.com/paper/2007.07683/) (2020)
- [Zero-Resource Cross-Domain Named Entity Recognition](https://mlsanity.com/paper/2002.05923/) (2020)
- [AdvPicker: Effectively Leveraging Unlabeled Data via Adversarial Discriminator for Cross-Lingual NER](https://mlsanity.com/paper/2106.02300/) (2021)
- [Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction](https://mlsanity.com/paper/2109.06798/) (2021)

## Built on by (most cited first, 1 of 1)

- [On the use of Silver Standard Data for Zero-shot Classification Tasks in Information Extraction](https://mlsanity.com/paper/2402.18061/) (2024)

## Abstract (arXiv)

Zero-shot cross-lingual information extraction(IE) aims at constructing an IE model for some low-resource target languages, given annotations exclusively in some rich-resource languages. Recent studies based on language-universal features have shown their effectiveness and are attracting increasing attention. However, prior work has neither explored the potential of establishing interactions between language-universal features and contextual representations nor incorporated features that can effectively model constituent span attributes and relationships between multiple spans. In this study, a syntax-augmented hierarchical interactive encoder (SHINE) is proposed to transfer cross-lingual IE knowledge. The proposed encoder is capable of interactively capturing complementary information between features and contextual information, to derive language-agnostic representations for various IE tasks. Concretely, a multi-level interaction network is designed to hierarchically interact the complementary information to strengthen domain adaptability. Besides, in addition to the well-studied syntax features of part-of-speech and dependency relation, a new syntax feature of constituency structure is introduced to model the constituent span information which is crucial for IE. Experiments across seven languages on three IE tasks and four benchmarks verify the effectiveness and generalization ability of the proposed method.