# Mapping the Course for Prompt-based Structured Prediction

arXiv: https://arxiv.org/abs/2508.15090 · HTML: https://mlsanity.com/paper/2508.15090/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 68 references that are in the map.
- First posted on arXiv 20 Aug 2025.

## Builds on (most cited first, 30 of 34)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision](https://mlsanity.com/paper/1904.12584/) (2019)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [A Survey of Confidence Estimation and Calibration in Large Language Models](https://mlsanity.com/paper/2311.08298/) (2023)
- [Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve](https://mlsanity.com/paper/2309.13638/) (2023)
- [A Survey on Semantic Parsing](https://mlsanity.com/paper/1812.00978/) (2018)
- [Joint Reasoning for Temporal and Causal Relations](https://mlsanity.com/paper/1906.04941/) (2019)
- [Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations](https://mlsanity.com/paper/1603.04351/) (2016)
- [Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction](https://mlsanity.com/paper/1909.05360/) (2019)
- [Autoregressive Structured Prediction with Language Models](https://mlsanity.com/paper/2210.14698/) (2022)
- [Identifying Morality Frames in Political Tweets using Relational Learning](https://mlsanity.com/paper/2109.04535/) (2021)
- [Question Answering is a Format; When is it Useful?](https://mlsanity.com/paper/1909.11291/) (2019)
- [Modeling Content and Context with Deep Relational Learning](https://mlsanity.com/paper/2010.10453/) (2020)
- ["You Are An Expert Linguistic Annotator": Limits of LLMs as Analyzers of Abstract Meaning Representation](https://mlsanity.com/paper/2310.17793/) (2023)
- [Structured Training for Neural Network Transition-Based Parsing](https://mlsanity.com/paper/1506.06158/) (2015)
- [Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions](https://mlsanity.com/paper/2311.07115/) (2023)
- [Neural Factor Graph Models for Cross-lingual Morphological Tagging](https://mlsanity.com/paper/1805.04570/) (2018)
- [Towards Few-Shot Identification of Morality Frames using In-Context Learning](https://mlsanity.com/paper/2302.02029/) (2023)
- [Using Domain Knowledge to Guide Dialog Structure Induction via Neural Probabilistic Soft Logic](https://mlsanity.com/paper/2403.17853/) (2024)

## Built on by (most cited first, 2 of 2)

- [Large Foundation Models for Trajectory Prediction in Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2509.10570/) (2025)
- [Verb-ICL: Rethinking In-Context Learning for Structured Prediction](https://mlsanity.com/paper/2610.04725/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated strong performance in a wide-range of language tasks without requiring task-specific fine-tuning. However, they remain prone to hallucinations and inconsistencies, and often struggle with complex reasoning, in part due to the limitations of autoregressive generation. We propose to address some of these issues, particularly for structured prediction, by combining LLMs with combinatorial inference to marry the predictive power of LLMs with the structural consistency provided by inference methods. We perform exhaustive experiments in an effort to understand which prompting strategies can best estimate confidence values for downstream symbolic inference, and find that, independent of prompting strategy, incorporating symbolic inference yields more consistent and accurate predictions than prompting alone. Finally, we show that calibration and fine-tuning with structured learning objectives further increases performance on challenging tasks, highlighting that structured learning remains valuable in the era of LLMs.