# Policy Gradient as a Proxy for Dynamic Oracles in Constituency Parsing

arXiv: https://arxiv.org/abs/1806.03290 · HTML: https://mlsanity.com/paper/1806.03290/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2018.

## Built on by (most cited first, 5 of 5)

- [BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions](https://mlsanity.com/paper/2406.15877/) (2024)
- [Quark: Controllable Text Generation with Reinforced Unlearning](https://mlsanity.com/paper/2205.13636/) (2022)
- [Grounding Language Models to Images for Multimodal Inputs and Outputs](https://mlsanity.com/paper/2301.13823/) (2023)
- [Discontinuous Grammar as a Foreign Language](https://mlsanity.com/paper/2110.10431/) (2021)
- [Improving Autoregressive Training with Dynamic Oracles](https://mlsanity.com/paper/2406.09393/) (2024)

## Abstract (arXiv)

Dynamic oracles provide strong supervision for training constituency parsers with exploration, but must be custom defined for a given parser's transition system. We explore using a policy gradient method as a parser-agnostic alternative. In addition to directly optimizing for a tree-level metric such as F1, policy gradient has the potential to reduce exposure bias by allowing exploration during training; moreover, it does not require a dynamic oracle for supervision. On four constituency parsers in three languages, the method substantially outperforms static oracle likelihood training in almost all settings. For parsers where a dynamic oracle is available (including a novel oracle which we define for the transition system of Dyer et al. 2016), policy gradient typically recaptures a substantial fraction of the performance gain afforded by the dynamic oracle.