mlsanityData to 14 Oct 2026

Instruct and Extract: Instruction Tuning for On-Demand Information Extraction

arXiv 2310.16040 ↗First posted 24 Oct 2023cs.CL cs.AIcited by 36

Not on the map. Why?

Abstract (arXiv)

Large language models with instruction-following capabilities open the door to a wider group of users. However, when it comes to information extraction - a classic task in natural language processing - most task-specific systems cannot align well with long-tail ad hoc extraction use cases for non-expert users. To address this, we propose a novel paradigm, termed On-Demand Information Extraction, to fulfill the personalized demands of real-world users. Our task aims to follow the instructions to extract the desired content from the associated text and present it in a structured tabular format. The table headers can either be user-specified or inferred contextually by the model. To facilitate research in this emerging area, we present a benchmark named InstructIE, inclusive of both automatically generated training data, as well as the human-annotated test set. Building on InstructIE, we further develop an On-Demand Information Extractor, ODIE. Comprehensive evaluations on our benchmark reveal that ODIE substantially outperforms the existing open-source models of similar size. Our code and dataset are released on https://github.com/yzjiao/On-Demand-IE.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Builds on 23 of its 48 references that are in the map. First posted on arXiv 24 Oct 2023.

Builds on

23 of 48 references.

References · 23

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  6. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  7. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  8. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  9. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  10. 2016Neural Architectures for Named Entity Recognitioncited by 550
  11. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 408
  12. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 397
11 more
  1. 2022Towards a Unified Multi-Dimensional Evaluator for Text Generationcited by 264
  2. 2018FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluationcited by 254
  3. 2019DocRED: A Large-Scale Document-Level Relation Extraction Datasetcited by 239
  4. 2023Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulnesscited by 130
  5. 2020MAVEN: A Massive General Domain Event Detection Datasetcited by 104
  6. 2019Meta-Learning with Dynamic-Memory-Based Prototypical Network for Few-Shot Event Detectioncited by 54
  7. 2020Joint Entity and Relation Extraction with Set Prediction Networkscited by 54
  8. 2021Text-to-Table: A New Way of Information Extractioncited by 47
  9. 2018Table-to-Text: Describing Table Region with Natural Languagecited by 41
  10. 2019Low-Resource Name Tagging Learned with Weakly Labeled Datacited by 32
  11. 2022Open-Vocabulary Argument Role Prediction for Event Extractioncited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 12; 2026: 14.

36 papers.

Sort:

By area:

  1. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  2. 2024SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literaturecited by 41
  3. 2024ADELIE: Aligning Large Language Models on Information Extractioncited by 24
  4. 2025Instruction-Tuning LLMs for Event Extraction with Annotation Guidelinescited by 18
  5. 2024A Simple but Effective Approach to Improve Structured Language Model Output for Information Extractioncited by 16
  6. 2025A Survey on Retrieval And Structuring Augmented Generation with Large Language Modelscited by 8
  7. 2024Scaling Knowledge Graph Construction through Synthetic Data Generation and Distillationcited by 6
  8. 2024IEPile: Unearthing Large-Scale Schema-Based Information Extraction Corpuscited by 5
  9. 2024Toward Reliable Ad-hoc Scientific Information Extraction: A Case Study on Two Materials Datasetscited by 5
  10. 2024Investigating Instruction Tuning Large Language Models on Graphscited by 5
  11. 2024BioMistral-NLU: Towards More Generalizable Medical Language Understanding through Instruction Tuningcited by 4
  12. 2025Surgical Action Planning with Large Language Modelscited by 4
24 more
  1. 2025Schema as Parameterized Tools for Universal Information Extractioncited by 3
  2. 2026ScheMatiQ: From Research Question to Structured Data through Interactive Schema Discoverycited by 3
  3. 2025Structuring the Unstructured: A Systematic Review of Text-to-Structure Generation for Agentic AI with a Universal Evaluation Frameworkcited by 2
  4. 2025Evaluating Structured Decoding for Text-to-Table Generation: Evidence from Three Datasetscited by 2
  5. 2026DTBench: A Synthetic Benchmark for Document-to-Table Extractioncited by 2
  6. 2025Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesiscited by 1
  7. 2025An agentic system with reinforcement-learned subsystem improvements for parsing form-like documentscited by 1
  8. 2025Zero-Shot Open-Schema Entity Structure Discoverycited by 1
  9. 2026ExStrucTiny: A Benchmark for Schema-Variable Structured Information Extraction from Document Imagescited by 1
  10. 2026Instructor-Aligned Knowledge Graphs for Personalized Learningcited by 1
  11. 2026GRASP: GRanularity-Aware Search Policy for Agentic RAGcited by 1
  12. 2024MetaKP: On-Demand Keyphrase Generationcited by 0
  13. 2025Low-Resource Fine-Tuning for Multi-Task Structured Information Extraction with a Billion-Parameter Instruction-Tuned Modelcited by 0
  14. 2025The Answer Lies Within: Self-Derived Rewards Enable Explainable Relation Extractioncited by 0
  15. 2025ToMMeR -- Efficient Entity Mention Detection from Large Language Modelscited by 0
  16. 2026LiveWeb-IE: A Benchmark For Online Web Information Extractioncited by 0
  17. 2026Relational Probing: LM-to-Graph Adaptation for Financial Predictioncited by 0
  18. 2026Closing the Loop: Unified 3D Scene Generation and Immersive Interaction via LLM-RL Couplingcited by 0
  19. 2026BoostTaxo: Zero-Shot Taxonomy Induction via Boosting-Style Agentic Reasoning and Constraint-Aware Calibrationcited by 0
  20. 2026Beyond Tables: Doc2DB-Bench for Relationally Faithful Document-to-Database Constructioncited by 0
  21. 2026From UNDRR Reports to Event Records: Schema-Constrained LLM Extraction of Georeferenced Disasterscited by 0
  22. 2026EAGER: Enhancing Generative Event Extraction via Reinforcement Learning with Verifiable Rewardscited by 0
  23. 2026From Weak Task Specifications to Scientific Extraction Agents: Optimizing Task Constructioncited by 0
  24. 2026Right Answers, Wrong States: Hidden Information Failures in Multi-Agent Collaborationcited by 0