# Torch-Struct: Deep Structured Prediction Library

arXiv: https://arxiv.org/abs/2002.00876 · HTML: https://mlsanity.com/paper/2002.00876/

## Key facts

- Cited by 40 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 3 Feb 2020.

## Built on by (most cited first, 30 of 40)

- [Gated Slot Attention for Efficient Linear-Time Sequence Modeling](https://mlsanity.com/paper/2409.07146/) (2024)
- [GFlowNet-EM for learning compositional latent variable models](https://mlsanity.com/paper/2302.06576/) (2023)
- [Sequence-to-Sequence Learning with Latent Neural Grammars](https://mlsanity.com/paper/2109.01135/) (2021)
- [PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols](https://mlsanity.com/paper/2104.13727/) (2021)
- [Model Criticism for Long-Form Text Generation](https://mlsanity.com/paper/2210.08444/) (2022)
- [Neural Bi-Lexicalized PCFG Induction](https://mlsanity.com/paper/2105.15021/) (2021)
- [Finding Dataset Shortcuts with Grammar Induction](https://mlsanity.com/paper/2210.11560/) (2022)
- [Semantic Role Labeling as Dependency Parsing: Exploring Latent Tree Structures Inside Arguments](https://mlsanity.com/paper/2110.06865/) (2021)
- [Nested Named Entity Recognition as Latent Lexicalized Constituency Parsing](https://mlsanity.com/paper/2203.04665/) (2022)
- [DomiKnowS: A Library for Integration of Symbolic Domain Knowledge in Deep Learning](https://mlsanity.com/paper/2108.12370/) (2021)
- [Headed-Span-Based Projective Dependency Parsing](https://mlsanity.com/paper/2108.04750/) (2021)
- [GenNI: Human-AI Collaboration for Data-Backed Text Generation](https://mlsanity.com/paper/2110.10185/) (2021)
- [Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs](https://mlsanity.com/paper/2205.00484/) (2022)
- [string2string: A Modern Python Library for String-to-String Algorithms](https://mlsanity.com/paper/2304.14395/) (2023)
- [Hierarchical Phrase-based Sequence-to-Sequence Learning](https://mlsanity.com/paper/2211.07906/) (2022)
- [PPT: Parsimonious Parser Transfer for Unsupervised Cross-Lingual Adaptation](https://mlsanity.com/paper/2101.11216/) (2021)
- [An Empirical Study of Compound PCFGs](https://mlsanity.com/paper/2103.02298/) (2021)
- [Masked Conditional Random Fields for Sequence Labeling](https://mlsanity.com/paper/2103.10682/) (2021)
- [Developmental Stage Classification of Embryos Using Two-Stream Neural Network with Linear-Chain Conditional Random Field](https://mlsanity.com/paper/2107.06360/) (2021)
- [A Latent-Variable Model for Intrinsic Probing](https://mlsanity.com/paper/2201.08214/) (2022)
- [A fast and sound tagging method for discontinuous named-entity recognition](https://mlsanity.com/paper/2409.16243/) (2024)
- [DeepCPCFG: Deep Learning and Context Free Grammars for End-to-End Information Extraction](https://mlsanity.com/paper/2103.05908/) (2021)
- [Recursive Bayesian Networks: Generalising and Unifying Probabilistic Context-Free Grammars and Dynamic Bayesian Networks](https://mlsanity.com/paper/2111.01853/) (2021)
- [Approximating CKY with Transformers](https://mlsanity.com/paper/2305.02386/) (2023)
- [Simple Hardware-Efficient PCFGs with Independent Left and Right Productions](https://mlsanity.com/paper/2310.14997/) (2023)
- [Scaling Structured Inference with Randomization](https://mlsanity.com/paper/2112.03638/) (2021)
- [Unbiased and Efficient Sampling of Dependency Trees](https://mlsanity.com/paper/2205.12621/) (2022)
- [TorchBench: Benchmarking PyTorch with High API Surface Coverage](https://mlsanity.com/paper/2304.14226/) (2023)
- [Multilingual Multiword Expression Identification Using Lateral Inhibition and Domain Adaptation](https://mlsanity.com/paper/2306.10419/) (2023)
- [SynJax: Structured Probability Distributions for JAX](https://mlsanity.com/paper/2308.03291/) (2023)

## Abstract (arXiv)

The literature on structured prediction for NLP describes a rich collection of distributions and algorithms over sequences, segmentations, alignments, and trees; however, these algorithms are difficult to utilize in deep learning frameworks. We introduce Torch-Struct, a library for structured prediction designed to take advantage of and integrate with vectorized, auto-differentiation based frameworks. Torch-Struct includes a broad collection of probabilistic structures accessed through a simple and flexible distribution-based API that connects to any deep learning model. The library utilizes batched, vectorized operations and exploits auto-differentiation to produce readable, fast, and testable code. Internally, we also include a number of general-purpose optimizations to provide cross-algorithm efficiency. Experiments show significant performance gains over fast baselines and case-studies demonstrate the benefits of the library. Torch-Struct is available at https://github.com/harvardnlp/pytorch-struct.