PaLM: A Hybrid Parser and Language Model
Not on the map. Why?
Abstract (arXiv)
We present PaLM, a hybrid parser and neural language model. Building on an RNN language model, PaLM adds an attention layer over text spans in the left context. An unsupervised constituency parser can be derived from its attention weights, using a greedy decoding algorithm. We evaluate PaLM on language modeling, and empirically show that it outperforms strong baselines. If syntactic annotations are available, the attention component can be trained in a supervised manner, providing syntactically-informed representations of the context, and further improving language modeling performance.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Sep 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 3; 2022: 2; 2023: 3; 2024: 2; 2025: 1; 2026: 2.
13 papers.
Sort:
- 2022Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scalecited by 40
- 2023Advancing Medical Imaging with Language Models: A Journey from N-grams to ChatGPTcited by 28
- 2021Structural Guidance for Transformer Language Modelscited by 25
- 2023Pushdown Layers: Encoding Recursive Structure in Transformer Language Modelscited by 18
- 2021Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modelingcited by 10
- 2024Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Modelscited by 7
- 2023Learning Representations on Logs for AIOpscited by 6
- 2024Tree-Planted Transformers: Unidirectional Transformer Language Models with Implicit Syntactic Supervisioncited by 4
- 2021Co-training an Unsupervised Constituency Parser with Weak Supervisioncited by 3
- 2022Dependency-based Mixture Language Modelscited by 1
- 2025Vocab Diet: Reshaping the Vocabulary of LLMs via Vector Arithmeticcited by 1
- 2026A Unified Assessment of the Poverty of the Stimulus Argument for Neural Language Modelscited by 1
1 more
- 2026GiLT: Augmenting Transformer Language Models with Dependency Graphs15 May 2026
- 2026A Unified Assessment of the Poverty of the Stimulus Argument for Neural Language Models10 Feb 2026
- 2025Vocab Diet: Reshaping the Vocabulary of LLMs via Vector Arithmetic19 Oct 2025
- 2024Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models24 Jul 2024
- 2024Tree-Planted Transformers: Unidirectional Transformer Language Models with Implicit Syntactic Supervision20 Feb 2024
- 2023Pushdown Layers: Encoding Recursive Structure in Transformer Language Models29 Oct 2023
- 2023Learning Representations on Logs for AIOps18 Aug 2023
- 2023Advancing Medical Imaging with Language Models: A Journey from N-grams to ChatGPT11 Apr 2023
- 2022Dependency-based Mixture Language Models19 Mar 2022
- 2022Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale1 Mar 2022
- 2021Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modeling15 Dec 2021
- 2021Co-training an Unsupervised Constituency Parser with Weak Supervision5 Oct 2021
1 more
- 2021Structural Guidance for Transformer Language Models30 Jul 2021
- 2022Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scalecited by 40
- 2023Advancing Medical Imaging with Language Models: A Journey from N-grams to ChatGPTcited by 28
- 2021Structural Guidance for Transformer Language Modelscited by 25
- 2023Pushdown Layers: Encoding Recursive Structure in Transformer Language Modelscited by 18
- 2021Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modelingcited by 10
- 2024Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Modelscited by 7
- 2023Learning Representations on Logs for AIOpscited by 6
- 2024Tree-Planted Transformers: Unidirectional Transformer Language Models with Implicit Syntactic Supervisioncited by 4
- 2021Co-training an Unsupervised Constituency Parser with Weak Supervisioncited by 3
- 2022Dependency-based Mixture Language Modelscited by 1
- 2025Vocab Diet: Reshaping the Vocabulary of LLMs via Vector Arithmeticcited by 1
- 2026A Unified Assessment of the Poverty of the Stimulus Argument for Neural Language Modelscited by 1
12 most cited of 13 All 13 (JSON)
- 2026GiLT: Augmenting Transformer Language Models with Dependency Graphs15 May 2026
- 2026A Unified Assessment of the Poverty of the Stimulus Argument for Neural Language Models10 Feb 2026
- 2025Vocab Diet: Reshaping the Vocabulary of LLMs via Vector Arithmetic19 Oct 2025
- 2024Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models24 Jul 2024
- 2024Tree-Planted Transformers: Unidirectional Transformer Language Models with Implicit Syntactic Supervision20 Feb 2024
- 2023Pushdown Layers: Encoding Recursive Structure in Transformer Language Models29 Oct 2023
- 2023Learning Representations on Logs for AIOps18 Aug 2023
- 2023Advancing Medical Imaging with Language Models: A Journey from N-grams to ChatGPT11 Apr 2023
- 2022Dependency-based Mixture Language Models19 Mar 2022
- 2022Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale1 Mar 2022
- 2021Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modeling15 Dec 2021
- 2021Co-training an Unsupervised Constituency Parser with Weak Supervision5 Oct 2021
12 newest of 13 All 13 (JSON)