# Mimic and Conquer: Heterogeneous Tree Structure Distillation for Syntactic NLP

arXiv: https://arxiv.org/abs/2009.07411 · HTML: https://mlsanity.com/paper/2009.07411/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2020.

## Built on by (most cited first, 15 of 15)

- [Scene Graph as Pivoting: Inference-time Image-free Unsupervised Multimodal Machine Translation with Visual Scene Hallucination](https://mlsanity.com/paper/2305.12256/) (2023)
- [On the Robustness of Aspect-based Sentiment Analysis: Rethinking Model, Data, and Training](https://mlsanity.com/paper/2304.09563/) (2023)
- [Entity-centered Cross-document Relation Extraction](https://mlsanity.com/paper/2210.16541/) (2022)
- [Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignment](https://mlsanity.com/paper/2305.12260/) (2023)
- [Constructing Holistic Spatio-Temporal Scene Graph for Video Semantic Role Labeling](https://mlsanity.com/paper/2308.05081/) (2023)
- [Conversational Semantic Role Labeling with Predicate-Oriented Latent Graph](https://mlsanity.com/paper/2210.03037/) (2022)
- [Revisiting Disentanglement and Fusion on Modality and Context in Conversational Multimodal Emotion Recognition](https://mlsanity.com/paper/2308.04502/) (2023)
- [Constructing Code-mixed Universal Dependency Forest for Unbiased Cross-lingual Relation Extraction](https://mlsanity.com/paper/2305.12258/) (2023)
- [ECQED: Emotion-Cause Quadruple Extraction in Dialogs](https://mlsanity.com/paper/2306.03969/) (2023)
- [TKDP: Threefold Knowledge-enriched Deep Prompt Tuning for Few-shot Named Entity Recognition](https://mlsanity.com/paper/2306.03974/) (2023)
- [Revisiting Conversation Discourse for Dialogue Disentanglement](https://mlsanity.com/paper/2306.03975/) (2023)
- [A Bi-directional Multi-hop Inference Model for Joint Dialog Sentiment Classification and Act Recognition](https://mlsanity.com/paper/2308.04424/) (2023)
- [Can We Use Probing to Better Understand Fine-tuning and Knowledge Distillation of the BERT NLU?](https://mlsanity.com/paper/2301.11688/) (2023)
- [DialogRE^C+: An Extension of DialogRE to Investigate How Much Coreference Helps Relation Extraction in Dialogs](https://mlsanity.com/paper/2308.04498/) (2023)
- [ExpertNet: A Symbiosis of Classification and Clustering](https://mlsanity.com/paper/2201.06344/) (2022)

## Abstract (arXiv)

Syntax has been shown useful for various NLP tasks, while existing work mostly encodes singleton syntactic tree using one hierarchical neural network. In this paper, we investigate a simple and effective method, Knowledge Distillation, to integrate heterogeneous structure knowledge into a unified sequential LSTM encoder. Experimental results on four typical syntax-dependent tasks show that our method outperforms tree encoders by effectively integrating rich heterogeneous structure syntax, meanwhile reducing error propagation, and also outperforms ensemble methods, in terms of both the efficiency and accuracy.