mlsanityData to 7 Oct 2026

TrICy: Trigger-guided Data-to-text Generation with Intent aware Attention-Copy

arXiv 2402.01714 ↗First posted 25 Jan 2024cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Data-to-text (D2T) generation is a crucial task in many natural language understanding (NLU) applications and forms the foundation of task-oriented dialog systems. In the context of conversational AI solutions that can work directly with local data on the user's device, architectures utilizing large pre-trained language models (PLMs) are impractical for on-device deployment due to a high memory footprint. To this end, we propose TrICy, a novel lightweight framework for an enhanced D2T task that generates text sequences based on the intent in context and may further be guided by user-provided triggers. We leverage an attention-copy mechanism to predict out-of-vocabulary (OOV) words accurately. Performance analyses on E2E NLG dataset (BLEU: 66.43%, ROUGE-L: 70.14%), WebNLG dataset (BLEU: Seen 64.08%, Unseen 52.35%), and our Custom dataset related to text messaging applications, showcase our architecture's effectiveness. Moreover, we show that by leveraging an optional trigger input, data-to-text generation quality increases significantly and achieves the new SOTA score of 69.29% BLEU for E2E NLG. Furthermore, our analyses show that TrICy achieves at least 24% and 3% improvement in BLEU and METEOR respectively over LLMs like GPT-3, ChatGPT, and Llama 2. We also demonstrate that in some scenarios, performance improvement due to triggers is observed even when they are absent in training.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 63 references that are in the map. First posted on arXiv 25 Jan 2024.

Builds on

35 of 63 references.

References · 35

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  10. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  11. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  12. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 660
23 more
  1. 2017The E2E Dataset: New Challenges For End-to-End Generationcited by 252
  2. 2016Incorporating Copying Mechanism in Sequence-to-Sequence Learningcited by 217
  3. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
  4. 2021A Recipe For Arbitrary Text Style Transfer with Large Language Modelscited by 107
  5. 2020Text-to-Text Pre-Training for Data-to-Text Taskscited by 104
  6. 2019CopyMTL: Copy Mechanism for Joint Extraction of Entities and Relations with Multi-Task Learningcited by 47
  7. 2019Data-to-text Generation with Entity Modelingcited by 45
  8. 2021A Closer Look at How Fine-tuning Changes BERTcited by 42
  9. 2020Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelitycited by 35
  10. 2019Pragmatically Informative Text Generationcited by 32
  11. 2023DirectGPT: A Direct Manipulation Interface to Interact with Large Language Modelscited by 32
  12. 2015What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignmentcited by 31
  13. 2020Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphscited by 30
  14. 2018Augmenting Neural Response Generation with Context-Aware Topical Attentioncited by 27
  15. 2023Using Large Language Models for Zero-Shot Natural Language Generation from Knowledge Graphscited by 24
  16. 2021Control Prefixes for Parameter-Efficient Text Generationcited by 23
  17. 2019Key Fact as Pivot: A Two-Stage Model for Low Resource Table-to-Text Generationcited by 20
  18. 2018Structure-Infused Copy Mechanisms for Abstractive Summarizationcited by 17
  19. 2023Evaluating Generative Models for Graph-to-Text Generationcited by 16
  20. 2021Stage-wise Fine-tuning for Graph-to-Text Generationcited by 11
  21. 2020Template Controllable keywords-to-text Generationcited by 3
  22. 2022TextBox 2.0: A Text Generation Library with Pre-trained Language Modelscited by 3
  23. 2020CopyNext: Explicit Span Copying and Alignment in Sequence to Sequence Modelscited by 1

Built on by

Not cited yet.