# Unified Text Structuralization with Instruction-tuned Language Models

arXiv: https://arxiv.org/abs/2303.14956 · HTML: https://mlsanity.com/paper/2303.14956/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 53 references that are in the map.
- First posted on arXiv 27 Mar 2023.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Solving Quantitative Reasoning Problems with Language Models](https://mlsanity.com/paper/2206.14858/) (2022)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [A Systematic Evaluation of Large Language Models of Code](https://mlsanity.com/paper/2202.13169/) (2022)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019)
- [End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures](https://mlsanity.com/paper/1601.00770/) (2016)
- [Graph Convolution over Pruned Dependency Trees Improves Relation Extraction](https://mlsanity.com/paper/1809.10185/) (2018)
- [The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives](https://mlsanity.com/paper/1909.01380/) (2019)
- [MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning](https://mlsanity.com/paper/2212.10773/) (2022)
- [Data-to-Text Generation with Content Selection and Planning](https://mlsanity.com/paper/1809.00582/) (2018)
- [A General Framework for Information Extraction using Dynamic Span Graphs](https://mlsanity.com/paper/1904.03296/) (2019)
- [Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing](https://mlsanity.com/paper/2210.13669/) (2022)
- [Packed Levitated Marker for Entity and Relation Extraction](https://mlsanity.com/paper/2109.06067/) (2021)
- [Text-to-Table: A New Way of Information Extraction](https://mlsanity.com/paper/2109.02707/) (2021)
- [A Frustratingly Easy Approach for Entity and Relation Extraction](https://mlsanity.com/paper/2010.12812/) (2020)
- [Span Model for Open Information Extraction on Accurate Corpus](https://mlsanity.com/paper/1901.10879/) (2019)
- [Adversarial training for multi-context joint entity and relation extraction](https://mlsanity.com/paper/1808.06876/) (2018)
- [WikiTableT: A Large-Scale Data-to-Text Dataset for Generating Wikipedia Article Sections](https://mlsanity.com/paper/2012.14919/) (2020)
- [Open Event Extraction from Online Text using a Generative Adversarial Network](https://mlsanity.com/paper/1908.09246/) (2019)

## Built on by (most cited first, 8 of 8)

- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges](https://mlsanity.com/paper/2406.11903/) (2024)
- [FM-OV3D: Foundation Model-based Cross-modal Knowledge Blending for Open-Vocabulary 3D Detection](https://mlsanity.com/paper/2312.14465/) (2023)
- [Structsum Generation for Faster Text Comprehension](https://mlsanity.com/paper/2401.06837/) (2024)
- [DocTabQA: Answering Questions from Long Documents Using Tables](https://mlsanity.com/paper/2408.11490/) (2024)
- [InfoDiffusion: Information Entropy Aware Diffusion Process for Non-Autoregressive Text Generation](https://mlsanity.com/paper/2310.11976/) (2023)
- [LLM driven Text-to-Table Generation through Sub-Tasks Guidance and Iterative Refinement](https://mlsanity.com/paper/2508.08653/) (2025)
- [Beyond Isolated Dots: Benchmarking Structured Table Construction as Deep Knowledge Extraction](https://mlsanity.com/paper/2507.16271/) (2025)

## Abstract (arXiv)

Text structuralization is one of the important fields of natural language processing (NLP) consists of information extraction (IE) and structure formalization. However, current studies of text structuralization suffer from a shortage of manually annotated high-quality datasets from different domains and languages, which require specialized professional knowledge. In addition, most IE methods are designed for a specific type of structured data, e.g., entities, relations, and events, making them hard to generalize to others. In this work, we propose a simple and efficient approach to instruct large language model (LLM) to extract a variety of structures from texts. More concretely, we add a prefix and a suffix instruction to indicate the desired IE task and structure type, respectively, before feeding the text into a LLM. Experiments on two LLMs show that this approach can enable language models to perform comparable with other state-of-the-art methods on datasets of a variety of languages and knowledge, and can generalize to other IE sub-tasks via changing the content of instruction. Another benefit of our approach is that it can help researchers to build datasets in low-source and domain-specific scenarios, e.g., fields in finance and law, with low cost.