mlsanityData to 14 Oct 2026

In Tree Structure Should Sentence Be Generated

arXiv 2406.14189 ↗First posted 20 Jun 2024cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Generative models reliant on sequential autoregression have been at the forefront of language generation for an extensive period, particularly following the introduction of widely acclaimed transformers. Despite its excellent performance, there are always some issues that we face today. For example, problems such as hallucinations and getting trapped in a logic loop may occur. To enhance the performance of existing systems, this paper introduces a new method for generating sequences in natural language, which involves generating the targeted sentence in a tree-traversing order. The paper includes an illustration of the theoretical basis and validity of the approach, as well as a comparison of its fundamentals with the diffusion model in graphic generation. Finally, a module called SenTree is introduced for generating an approximating binary tree. It is already available at https://github.com/arklyg/sentree. Additionally, a joint training framework based on this approach is proposed, incorporating the intrinsics of generative adversarial networks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 38 references that are in the map. First posted on arXiv 20 Jun 2024.

Builds on

22 of 38 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2018Deep contextualized word representationscited by 2,290
  9. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  10. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  11. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  12. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
10 more
  1. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 530
  2. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 457
  3. 2016Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Taskscited by 216
  4. 2017What do Neural Machine Translation Models Learn about Morphology?cited by 166
  5. 2018Dissecting Contextual Word Embeddings: Architecture and Representationcited by 146
  6. 2019Attention Interpretability Across NLP Taskscited by 71
  7. 2020How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Maskingcited by 63
  8. 2017Neural Language Modeling by Jointly Learning Syntax and Lexiconcited by 57
  9. 2015Measuring Word Significance using Distributed Representations of Wordscited by 21
  10. 2023Assessing Word Importance Using Models Trained for Semantic Taskscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
  2. 2024A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models
  3. 2024Energy-Based Diffusion Language Models for Text Generation
  4. 2025Discrete Diffusion Models for Language Generation
  5. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers
  6. 2025Flatten Graphs as Sequences: Transformers are Scalable Graph Generators
  7. 2025DLM-One: Diffusion Language Models for One-Step Sequence Generation
  8. 2019Non-Monotonic Sequential Text Generation