# Globally Normalized Transition-Based Neural Networks

arXiv: https://arxiv.org/abs/1603.06042 · HTML: https://mlsanity.com/paper/1603.06042/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2016.

## Built on by (most cited first, 26 of 26)

- [Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attacks](https://mlsanity.com/paper/2110.12976/) (2021)
- [On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgetting](https://mlsanity.com/paper/2206.00761/) (2022)
- [Text Generation by Learning from Demonstrations](https://mlsanity.com/paper/2009.07839/) (2020)
- [Identifying Morality Frames in Political Tweets using Relational Learning](https://mlsanity.com/paper/2109.04535/) (2021)
- [Evaluating Distributional Distortion in Neural Language Modeling](https://mlsanity.com/paper/2203.12788/) (2022)
- [Learning Logic Rules for Document-level Relation Extraction](https://mlsanity.com/paper/2111.05407/) (2021)
- [Fast and Reliable Evaluation of Adversarial Robustness with Minimum-Margin Attack](https://mlsanity.com/paper/2206.07314/) (2022)
- [Energy-Based Models for Code Generation under Compilability Constraints](https://mlsanity.com/paper/2106.04985/) (2021)
- [Global Normalization for Streaming Speech Recognition in a Modular Framework](https://mlsanity.com/paper/2205.13674/) (2022)
- [Learning Beam Search Policies via Imitation Learning](https://mlsanity.com/paper/1811.00512/) (2018)
- [Time-series Generation by Contrastive Imitation](https://mlsanity.com/paper/2311.01388/) (2023)
- [Transformers as Graph-to-Graph Models](https://mlsanity.com/paper/2310.17936/) (2023)
- [Randomized Deep Structured Prediction for Discourse-Level Processing](https://mlsanity.com/paper/2101.10435/) (2021)
- [Detect and Defense Against Adversarial Examples in Deep Learning using Natural Scene Statistics and Adaptive Denoising](https://mlsanity.com/paper/2107.05780/) (2021)
- [EG-Booster: Explanation-Guided Booster of ML Evasion Attacks](https://mlsanity.com/paper/2108.13930/) (2021)
- [A Globally Normalized Neural Model for Semantic Parsing](https://mlsanity.com/paper/2106.03376/) (2021)
- [Mode recovery in neural autoregressive sequence modeling](https://mlsanity.com/paper/2106.05459/) (2021)
- [Efficient and Robust Classification for Sparse Attacks](https://mlsanity.com/paper/2201.09369/) (2022)
- [Algorithms for Weighted Pushdown Automata](https://mlsanity.com/paper/2210.06884/) (2022)
- [Exploring Energy-based Language Models with Different Architectures and Training Methods for Speech Recognition](https://mlsanity.com/paper/2305.12676/) (2023)
- [HOLMES: to Detect Adversarial Examples with Multiple Detectors](https://mlsanity.com/paper/2405.19956/) (2024)
- [Stealthy Multi-Task Adversarial Attacks](https://mlsanity.com/paper/2411.17936/) (2024)
- [Beyond Attack Success Rate: A Multi-Metric Evaluation of Adversarial Transferability in Medical Imaging Models](https://mlsanity.com/paper/2604.16532/) (2026)
- [Adversarial Stein Training for Graph Energy Models](https://mlsanity.com/paper/2108.12982/) (2021)
- [Towards a neural architecture of language: Deep learning versus logistics of access in neural architectures for compositional processing](https://mlsanity.com/paper/2210.10543/) (2022)
- [Globally Normalising the Transducer for Streaming Speech Recognition](https://mlsanity.com/paper/2307.10975/) (2023)

## Abstract (arXiv)

We introduce a globally normalized transition-based neural network model that achieves state-of-the-art part-of-speech tagging, dependency parsing and sentence compression results. Our model is a simple feed-forward neural network that operates on a task-specific transition system, yet achieves comparable or better accuracies than recurrent models. We discuss the importance of global as opposed to local normalization: a key insight is that the label bias problem implies that globally normalized models can be strictly more expressive than locally normalized models.