# Non-neural Models Matter: A Re-evaluation of Neural Referring Expression Generation Systems

arXiv: https://arxiv.org/abs/2203.08274 · HTML: https://mlsanity.com/paper/2203.08274/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 48 references that are in the map.
- First posted on arXiv 15 Mar 2022.

## Builds on (most cited first, 8 of 8)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Generation and Comprehension of Unambiguous Object Descriptions](https://mlsanity.com/paper/1511.02283/) (2015)
- [Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation](https://mlsanity.com/paper/1703.09902/) (2017)
- [Neural data-to-text generation: A comparison between pipeline and end-to-end architectures](https://mlsanity.com/paper/1908.09022/) (2019)
- [Findings of the E2E NLG Challenge](https://mlsanity.com/paper/1810.01170/) (2018)
- [NeuralREG: An end-to-end approach to referring expression generation](https://mlsanity.com/paper/1805.08093/) (2018)
- [Referring Expression Generation Using Entity Profiles](https://mlsanity.com/paper/1909.01528/) (2019)
- [What can Neural Referential Form Selectors Learn?](https://mlsanity.com/paper/2108.06806/) (2021)

## Built on by (most cited first, 7 of 7)

- [A Survey on Semantic Processing Techniques](https://mlsanity.com/paper/2310.18345/) (2023)
- [Models of reference production: How do they withstand the test of time?](https://mlsanity.com/paper/2307.14817/) (2023)
- [Pento-DIARef: A Diagnostic Dataset for Learning the Incremental Algorithm for Referring Expression Generation from Examples](https://mlsanity.com/paper/2305.15087/) (2023)
- [The Role of Explanatory Value in Natural Language Processing](https://mlsanity.com/paper/2209.06169/) (2022)
- [Assessing Neural Referential Form Selectors on a Realistic Multilingual Dataset](https://mlsanity.com/paper/2210.04828/) (2022)
- [Intrinsic Task-based Evaluation for Referring Expression Generation](https://mlsanity.com/paper/2402.07432/) (2024)
- [Computational Modelling of Plurality and Definiteness in Chinese Noun Phrases](https://mlsanity.com/paper/2403.04376/) (2024)

## Abstract (arXiv)

In recent years, neural models have often outperformed rule-based and classic Machine Learning approaches in NLG. These classic approaches are now often disregarded, for example when new neural models are evaluated. We argue that they should not be overlooked, since, for some tasks, well-designed non-neural approaches achieve better performance than neural ones. In this paper, the task of generating referring expressions in linguistic context is used as an example. We examined two very different English datasets (WEBNLG and WSJ), and evaluated each algorithm using both automatic and human evaluations. Overall, the results of these evaluations suggest that rule-based systems with simple rule sets achieve on-par or better performance on both datasets compared to state-of-the-art neural REG systems. In the case of the more realistic dataset, WSJ, a machine learning-based system with well-designed linguistic features performed best. We hope that our work can encourage researchers to consider non-neural models in future.