# Promptly Predicting Structures: The Return of Inference

arXiv: https://arxiv.org/abs/2401.06877 · HTML: https://mlsanity.com/paper/2401.06877/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 59 references that are in the map.
- First posted on arXiv 12 Jan 2024.

## Builds on (most cited first, 30 of 30)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [Surface Form Competition: Why the Highest Probability Answer Isn't Always Right](https://mlsanity.com/paper/2104.08315/) (2021)
- [Event Extraction by Answering (Almost) Natural Questions](https://mlsanity.com/paper/2004.13625/) (2020)
- [Template-Based Named Entity Recognition Using BART](https://mlsanity.com/paper/2106.01760/) (2021)
- [Coreference Resolution as Query-based Span Prediction](https://mlsanity.com/paper/1911.01746/) (2019)
- [Mitigating Label Biases for In-context Learning](https://mlsanity.com/paper/2305.19148/) (2023)
- [Iteratively Prompt Pre-trained Language Models for Chain of Thought](https://mlsanity.com/paper/2203.08383/) (2022)
- [Code4Struct: Code Generation for Few-Shot Event Structure Prediction](https://mlsanity.com/paper/2210.12810/) (2022)
- [A Logic-Driven Framework for Consistency of Neural Models](https://mlsanity.com/paper/1909.00126/) (2019)
- [Prototypical Calibration for Few-shot Learning of Language Models](https://mlsanity.com/paper/2205.10183/) (2022)
- [UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training](https://mlsanity.com/paper/2202.12359/) (2022)
- [Large-Scale QA-SRL Parsing](https://mlsanity.com/paper/1805.05377/) (2018)
- [General-Purpose Question-Answering with Macaw](https://mlsanity.com/paper/2109.02593/) (2021)
- [Prompting Language Models for Linguistic Structure](https://mlsanity.com/paper/2211.07830/) (2022)
- [QADiscourse -- Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines](https://mlsanity.com/paper/2010.02815/) (2020)
- [Question Answering is a Format; When is it Useful?](https://mlsanity.com/paper/1909.11291/) (2019)
- [Asking It All: Generating Contextualized Questions for any Semantic Role](https://mlsanity.com/paper/2109.04832/) (2021)
- [Design Choices for Crowdsourcing Implicit Discourse Relations: Revealing the Biases Introduced by Task Design](https://mlsanity.com/paper/2304.00815/) (2023)
- [QASem Parsing: Text-to-text Modeling of QA-based Semantics](https://mlsanity.com/paper/2205.11413/) (2022)
- [Global Constraints with Prompting for Zero-Shot Event Argument Classification](https://mlsanity.com/paper/2302.04459/) (2023)

## Built on by (most cited first, 4 of 4)

- [Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labeling](https://mlsanity.com/paper/2401.14556/) (2024)
- [ProRefine: Inference-Time Prompt Refinement with Textual Feedback](https://mlsanity.com/paper/2506.05305/) (2025)
- [Superlatives in Context: Modeling the Implicit Semantics of Superlatives](https://mlsanity.com/paper/2405.20967/) (2024)
- [Mapping the Course for Prompt-based Structured Prediction](https://mlsanity.com/paper/2508.15090/) (2025)

## Abstract (arXiv)

Prompt-based methods have been used extensively across NLP to build zero- and few-shot label predictors. Many NLP tasks are naturally structured: that is, their outputs consist of multiple labels which constrain each other. Annotating data for such tasks can be cumbersome. Can the promise of the prompt-based paradigm be extended to such structured outputs? In this paper, we present a framework for constructing zero- and few-shot linguistic structure predictors. Our key insight is that we can use structural constraints -- and combinatorial inference derived from them -- to filter out inconsistent structures predicted by large language models. We instantiated this framework on two structured prediction tasks, and five datasets. Across all cases, our results show that enforcing consistency not only constructs structurally valid outputs, but also improves performance over the unconstrained variants.