mlsanityData to 7 Oct 2026

Retrofitting Structure-aware Transformer Language Model for End Tasks

arXiv 2009.07408 ↗First posted 16 Sep 2020cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

We consider retrofitting structure-aware Transformer-based language model for facilitating end tasks by proposing to exploit syntactic distance to encode both the phrasal constituency and dependency connection into the language model. A middle-layer structural learning strategy is leveraged for structure integration, accomplished with main semantic task training under multi-task learning scheme. Experimental results show that the retrofitted structure-aware Transformer language model achieves improved perplexity, meanwhile inducing accurate syntactic phrases. By performing structure-aware fine-tuning, our model achieves significant improvements for both semantic- and syntactic-dependent tasks.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 1; 2023: 19; 2024: 2.

24 papers.

Sort:

By area:

  1. 2023Reasoning Implicit Sentiment with Chain-of-Thought Promptingcited by 79
  2. 2023LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Modelcited by 43
  3. 2023Scene Graph as Pivoting: Inference-time Image-free Unsupervised Multimodal Machine Translation with Visual Scene Hallucinationcited by 27
  4. 2023On the Robustness of Aspect-based Sentiment Analysis: Rethinking Model, Data, and Trainingcited by 20
  5. 2023Generating Visual Spatial Description via Holistic 3D Scene Understandingcited by 16
  6. 2021Modeling Hierarchical Structures with Continuous Recursive Neural Networkscited by 15
  7. 2023Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignmentcited by 15
  8. 2023Constructing Holistic Spatio-Temporal Scene Graph for Video Semantic Role Labelingcited by 15
  9. 2022Conversational Semantic Role Labeling with Predicate-Oriented Latent Graphcited by 14
  10. 2023Revisiting Disentanglement and Fusion on Modality and Context in Conversational Multimodal Emotion Recognitioncited by 13
  11. 2023Constructing Code-mixed Universal Dependency Forest for Unbiased Cross-lingual Relation Extractioncited by 6
  12. 2023Beam Tree Recursive Cellscited by 6
12 more
  1. 2021Cooperative Self-training of Machine Reading Comprehensioncited by 5
  2. 2023Efficient Beam Tree Recursioncited by 4
  3. 2024Learning Shortcuts: On the Misleading Promise of NLU in Language Modelscited by 4
  4. 2023ECQED: Emotion-Cause Quadruple Extraction in Dialogscited by 2
  5. 2023TKDP: Threefold Knowledge-enriched Deep Prompt Tuning for Few-shot Named Entity Recognitioncited by 2
  6. 2023Revisiting Conversation Discourse for Dialogue Disentanglementcited by 2
  7. 2023A Bi-directional Multi-hop Inference Model for Joint Dialog Sentiment Classification and Act Recognitioncited by 2
  8. 2023Retrofitting Light-weight Language Models for Emotions using Supervised Contrastive Learningcited by 2
  9. 2023DialogRE^C+: An Extension of DialogRE to Investigate How Much Coreference Helps Relation Extraction in Dialogscited by 1
  10. 2023Syntactic Fusion: Enhancing Aspect-Level Sentiment Analysis Through Multi-Tree Graph Integrationcited by 0
  11. 2023Dynamic Syntax Mapping: A New Approach to Unsupervised Syntax Parsingcited by 0
  12. 2024Grammar Induction from Visual, Speech and Textcited by 0