mlsanityData to 14 Oct 2026

Explicit Syntactic Guidance for Neural Text Generation

arXiv 2306.11485 ↗First posted 20 Jun 2023cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Most existing text generation models follow the sequence-to-sequence paradigm. Generative Grammar suggests that humans generate natural language texts by learning language grammar. We propose a syntax-guided generation schema, which generates the sequence guided by a constituency parse tree in a top-down direction. The decoding process can be decomposed into two parts: (1) predicting the infilling texts for each constituent in the lexicalized syntax context given the source sentence; (2) mapping and expanding each constituent to construct the next-level syntax context. Accordingly, we propose a structural beam search method to find possible syntax structures hierarchically. Experiments on paraphrase generation and machine translation show that the proposed method outperforms autoregressive baselines, while also demonstrating effectiveness in terms of interpretability, controllability, and diversity.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 37 of its 72 references that are in the map. First posted on arXiv 20 Jun 2023.

Builds on

37 of 72 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  8. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  9. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  10. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  11. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  12. 2021Ethical and social risks of harm from Language Modelscited by 966
25 more
  1. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  2. 2016Sequence-Level Knowledge Distillationcited by 801
  3. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  4. 2019Revealing the Dark Secrets of BERTcited by 301
  5. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 286
  6. 2021Societal Biases in Language Generation: Progress and Challengescited by 202
  7. 2018Constituency Parsing with a Self-Attentive Encodercited by 199
  8. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 138
  9. 2018Multilingual Constituency Parsing with Self-Attention and Pre-Trainingcited by 117
  10. 2016Recurrent Neural Network Grammarscited by 109
  11. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 61
  12. 2021The paradox of the compositionality of natural language: a neural machine translation case studycited by 59
  13. 2017A causal framework for explaining the predictions of black-box sequence-to-sequence modelscited by 58
  14. 2020Syntax-guided Controlled Generation of Paraphrasescited by 47
  15. 2019Controllable Paraphrase Generation with a Syntactic Exemplarcited by 44
  16. 2021On Compositional Generalization of Neural Machine Translationcited by 34
  17. 2022Quality Controlled Paraphrase Generationcited by 31
  18. 2017Towards String-to-Tree Neural Machine Translationcited by 19
  19. 2019Towards Understanding Neural Machine Translation with Word Importancecited by 19
  20. 2018A Tree-based Decoder for Neural Machine Translationcited by 18
  21. 2017Learning to Parse and Translate Improves Neural Machine Translationcited by 16
  22. 2018Global Encoding for Abstractive Summarizationcited by 16
  23. 2017Modeling Source Syntax for Neural Machine Translationcited by 12
  24. 2017Syntax-Directed Attention for Neural Machine Translationcited by 10
  25. 2020Interpretable NLG for Task-oriented Dialogue Systems with Heterogeneous Rendering Machinescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 1; 2026: 2.

8 papers.

  1. 2024Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practicescited by 20
  2. 2024Compressing Long Context for Enhancing RAG with AMR-based Concept Distillationcited by 16
  3. 2024Retrieval is Accurate Generationcited by 10
  4. 2023Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoderscited by 6
  5. 2024Cross-domain Chinese Sentence Pattern Parsingcited by 0
  6. 2025Syntax-Guided Diffusion Language Models with User-Integrated Personalizationcited by 0
  7. 2026CoCR-RAG: Enhancing Retrieval-Augmented Generation in Web Q&A via Concept-oriented Context Reconstructioncited by 0
  8. 2026Top-down string-to-dependency Neural Machine Translationcited by 0