mlsanityData to 7 Oct 2026

AmbigNLG: Addressing Task Ambiguity in Instruction for NLG

arXiv 2402.17717 ↗First posted 27 Feb 2024cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

We introduce AmbigNLG, a novel task designed to tackle the challenge of task ambiguity in instructions for Natural Language Generation (NLG). Ambiguous instructions often impede the performance of Large Language Models (LLMs), especially in complex NLG tasks. To tackle this issue, we propose an ambiguity taxonomy that categorizes different types of instruction ambiguities and refines initial instructions with clearer specifications. Accompanying this task, we present AmbigSNI-NLG, a dataset comprising 2,500 instances annotated to facilitate research in AmbigNLG. Through comprehensive experiments with state-of-the-art LLMs, we demonstrate that our method significantly enhances the alignment of generated text with user expectations, achieving up to a 15.02-point increase in ROUGE scores. Our findings highlight the critical importance of addressing task ambiguity to fully harness the capabilities of LLMs in NLG tasks. Furthermore, we confirm the effectiveness of our method in practical settings involving interactive ambiguity mitigation with users, underscoring the benefits of leveraging LLMs for interactive clarification.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 38 of its 62 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

38 of 62 references.

References · 38

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  11. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  12. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
26 more
  1. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  2. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  3. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  4. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  5. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  6. 2018Probabilistic Model-Agnostic Meta-Learningcited by 254
  7. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
  8. 2022Is GPT-3 a Good Data Annotator?cited by 208
  9. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 124
  10. 2020Enabling Language Models to Fill in the Blankscited by 121
  11. 2017Controllable Abstractive Summarizationcited by 110
  12. 2023Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Taskscited by 102
  13. 2023Guiding Large Language Models via Directional Stimulus Promptingcited by 94
  14. 2023Batch Prompting: Efficient Inference with Large Language Model APIscited by 84
  15. 2019Mixture Models for Diverse Machine Translation: Tricks of the Tradecited by 77
  16. 2022On Improving Summarization Factual Consistency from Natural Language Feedbackcited by 38
  17. 2022Active Learning Helps Pretrained Models Learn the Intended Taskcited by 33
  18. 2023Benchmarking and Improving Text-to-SQL Generation under Ambiguitycited by 31
  19. 2023Less is More for Long Document Summary Evaluation by LLMscited by 28
  20. 2023Discrete Prompt Optimization via Constrained Generation for Zero-shot Re-rankercited by 20
  21. 2021Comparative Opinion Summarization via Collaborative Decodingcited by 18
  22. 2023Interactive-Chain-Prompting: Ambiguity Resolution for Crosslingual Conditional Generation with Interactioncited by 16
  23. 2022Boosting Natural Language Generation from Instructions with Meta-Learningcited by 15
  24. 2020Fact-based Text Editingcited by 14
  25. 2022AutoTemplate: A Simple Recipe for Lexically Constrained Text Generationcited by 6
  26. 2023XATU: A Fine-grained Instruction-based Benchmark for Explainable Text Updatescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 4.

10 papers.

  1. 2025Ambig-SWE: Interactive Agents to Overcome Underspecificity in Software Engineeringcited by 28
  2. 2025A Survey on Large Language Models for Automated Planningcited by 17
  3. 2025Who Relies More on World Knowledge and Bias for Syntactic Ambiguity Resolution: Humans or LLMs?cited by 3
  4. 2026Asking What Matters: Reward-Driven Clarification for Software Engineering Taskscited by 3
  5. 2025Correct-Detect: Balancing Performance and Ambiguity Through the Lens of Coreference Resolution in LLMscited by 1
  6. 2025Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code Generationcited by 1
  7. 2026The Architecture of Errors: From Universal Impossibility to Patch-Local LLM Reliabilitycited by 1
  8. 2025Next-Token Prediction Should be Ambiguity-Sensitive: A Meta-Learning Perspectivecited by 0
  9. 2026DRIP-R: A Benchmark for Decision-Making and Reasoning Under Real-World Policy Ambiguity in the Retail Domaincited by 0
  10. 2026Characterizing Overconfident Failure in LLM-Based Code Generationcited by 0