# Non-Autoregressive Sentence Ordering

arXiv: https://arxiv.org/abs/2310.12640 · HTML: https://mlsanity.com/paper/2310.12640/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 19 Oct 2023.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1711.02281/) (2017)
- [A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories](https://mlsanity.com/paper/1604.01696/) (2016)
- [Levenshtein Transformer](https://mlsanity.com/paper/1905.11006/) (2019)
- [Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement](https://mlsanity.com/paper/1802.06901/) (2018)
- [Visual Storytelling](https://mlsanity.com/paper/1604.03968/) (2016)
- [Insertion Transformer: Flexible Sequence Generation via Insertion Operations](https://mlsanity.com/paper/1902.03249/) (2019)
- [Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2106.05093/) (2021)
- [Non-Autoregressive Machine Translation with Auxiliary Regularization](https://mlsanity.com/paper/1902.10245/) (2019)
- [Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input](https://mlsanity.com/paper/1812.09664/) (2018)
- [Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1911.08717/) (2019)
- [Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering Information](https://mlsanity.com/paper/1911.02215/) (2019)
- [BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining](https://mlsanity.com/paper/2012.15525/) (2020)
- [Hint-Based Training for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/1909.06708/) (2019)
- [KERMIT: Generative Insertion-Based Modeling for Sequences](https://mlsanity.com/paper/1906.01604/) (2019)
- [Deep Attentive Ranking Networks for Learning to Order Sentences](https://mlsanity.com/paper/2001.00056/) (2019)
- [Topological Sort for Sentence Ordering](https://mlsanity.com/paper/2005.00432/) (2020)
- [Neural Sentence Ordering](https://mlsanity.com/paper/1607.06952/) (2016)
- [End-to-End Neural Sentence Ordering Using Pointer Network](https://mlsanity.com/paper/1611.04953/) (2016)
- [Non-Autoregressive Dialog State Tracking](https://mlsanity.com/paper/2002.08024/) (2020)
- [Is Everything in Order? A Simple Way to Order Sentences](https://mlsanity.com/paper/2104.07064/) (2021)
- [Paper Abstract Writing through Editing Mechanism](https://mlsanity.com/paper/1805.06064/) (2018)
- [Non-Autoregressive Math Word Problem Solver with Unified Tree Structure](https://mlsanity.com/paper/2305.04556/) (2023)

## Abstract (arXiv)

Existing sentence ordering approaches generally employ encoder-decoder frameworks with the pointer net to recover the coherence by recurrently predicting each sentence step-by-step. Such an autoregressive manner only leverages unilateral dependencies during decoding and cannot fully explore the semantic dependency between sentences for ordering. To overcome these limitations, in this paper, we propose a novel Non-Autoregressive Ordering Network, dubbed textitNAON, which explores bilateral dependencies between sentences and predicts the sentence for each position in parallel. We claim that the non-autoregressive manner is not just applicable but also particularly suitable to the sentence ordering task because of two peculiar characteristics of the task: 1) each generation target is in deterministic length, and 2) the sentences and positions should match exclusively. Furthermore, to address the repetition issue of the naive non-autoregressive Transformer, we introduce an exclusive loss to constrain the exclusiveness between positions and sentences. To verify the effectiveness of the proposed model, we conduct extensive experiments on several common-used datasets and the experimental results show that our method outperforms all the autoregressive approaches and yields competitive performance compared with the state-of-the-arts. The codes are available at: urlhttps://github.com/steven640pixel/nonautoregressive-sentence-ordering.