# Inroads to a Structured Data Natural Language Bijection and the role of LLM annotation

arXiv: https://arxiv.org/abs/2401.07190 · HTML: https://mlsanity.com/paper/2401.07190/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 37 references that are in the map.
- First posted on arXiv 14 Jan 2024.

## Builds on (most cited first, 15 of 15)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://mlsanity.com/paper/2305.02301/) (2023)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning](https://mlsanity.com/paper/2111.10952/) (2021)
- [A Survey on Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2202.01110/) (2022)
- [Does Synthetic Data Generation of LLMs Help Clinical Text Mining?](https://mlsanity.com/paper/2303.04360/) (2023)
- [GO FIGURE: A Meta Evaluation of Factuality in Summarization](https://mlsanity.com/paper/2010.12834/) (2020)
- [Plan-then-Generate: Controlled Data-to-Text Generation via Planning](https://mlsanity.com/paper/2108.13740/) (2021)
- [Few-Shot Semantic Parsing with Language Models Trained On Code](https://mlsanity.com/paper/2112.08696/) (2021)
- [Control Prefixes for Parameter-Efficient Text Generation](https://mlsanity.com/paper/2110.08329/) (2021)
- [Using Large Language Models for Zero-Shot Natural Language Generation from Knowledge Graphs](https://mlsanity.com/paper/2307.07312/) (2023)
- [ReGen: Reinforcement Learning for Text and Knowledge Base Generation using Pretrained Language Models](https://mlsanity.com/paper/2108.12472/) (2021)

## Abstract (arXiv)

This work finds limited evidence supporting the theory that using multiple tasks with sequence-to-sequence transformer language models can improve performance on some metrics. In particular, the multi-task generalist t5-small outperforms the specialist t5-small with a F1 of 0.771 up from 0.692, which may point to underlying cross-task knowledge generalization. This further suggests that even with the same network, "re-using" the same data in a different way may lead to higher performance in some metrics. However, the inverse task alone is likely only an optimization strategy, since it does not yield a significant general improvement at the model sizes explored in this work. Also, adding ≈ 4500 LLM annotated records (interlaced with the 12800 WebNLG training records) does not substantially change automatic metric performance compared to the same t5-small model without the synthetic data. This may be due to a learning capacity bottleneck on account of model size, and decreases observed may be due to distributional differences in the corpora. Future research using larger models or human evaluation is required to more fully explain the mechanisms contributing to performance on these tasks.