# What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment

arXiv: https://arxiv.org/abs/1509.00838 · HTML: https://mlsanity.com/paper/1509.00838/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Sep 2015.

## Built on by (most cited first, 30 of 31)

- [Chart-to-Text: A Large-Scale Benchmark for Chart Summarization](https://mlsanity.com/paper/2203.06486/) (2022)
- [OpenCQA: Open-ended Question Answering with Charts](https://mlsanity.com/paper/2210.06628/) (2022)
- [Controlling Hallucinations at Word Level in Data-to-Text Generation](https://mlsanity.com/paper/2102.02810/) (2021)
- [Data-to-text Generation with Macro Planning](https://mlsanity.com/paper/2102.02723/) (2021)
- [Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View](https://mlsanity.com/paper/2102.08585/) (2021)
- [Focused Attention Improves Document-Grounded Generation](https://mlsanity.com/paper/2104.12714/) (2021)
- [Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation](https://mlsanity.com/paper/2105.14778/) (2021)
- [Data-to-text Generation with Variational Sequential Planning](https://mlsanity.com/paper/2202.13756/) (2022)
- [GenNI: Human-AI Collaboration for Data-Backed Text Generation](https://mlsanity.com/paper/2110.10185/) (2021)
- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)
- [What Makes Data-to-Text Generation Hard for Pretrained Language Models?](https://mlsanity.com/paper/2205.11505/) (2022)
- [MURMUR: Modular Multi-Step Reasoning for Semi-Structured Data-to-Text Generation](https://mlsanity.com/paper/2212.08607/) (2022)
- [A modular vision language navigation and manipulation framework for long horizon compositional tasks in indoor environment](https://mlsanity.com/paper/2101.07891/) (2021)
- [DiscrimLoss: A Universal Loss for Hard Samples and Incorrect Samples Discrimination](https://mlsanity.com/paper/2208.09884/) (2022)
- [XAlign: Cross-lingual Fact-to-Text Alignment and Generation for Low-Resource Languages](https://mlsanity.com/paper/2202.00291/) (2022)
- [Towards Table-to-Text Generation with Pretrained Language Model: A Table Structure Understanding and Text Deliberating Approach](https://mlsanity.com/paper/2301.02071/) (2023)
- [Regularization-based Pruning of Irrelevant Weights in Deep Neural Architectures](https://mlsanity.com/paper/2204.04977/) (2022)
- [Understanding Game-Playing Agents with Natural Language Annotations](https://mlsanity.com/paper/2204.07531/) (2022)
- [From Multimodal Perception to Strategic Reasoning: A Survey on AI-Generated Game Commentary](https://mlsanity.com/paper/2506.17294/) (2025)
- [Adversarial Machine Learning in Text Analysis and Generation](https://mlsanity.com/paper/2101.08675/) (2021)
- [WikiTableEdit: A Benchmark for Table Editing by Natural Language Instruction](https://mlsanity.com/paper/2403.02962/) (2024)
- [A Library for Automatic Natural Language Generation of Spanish Texts](https://mlsanity.com/paper/2405.17280/) (2024)
- [XF2T: Cross-lingual Fact-to-Text Generation for Low-Resource Languages](https://mlsanity.com/paper/2209.11252/) (2022)
- [CB2: Collaborative Natural Language Interaction Research Platform](https://mlsanity.com/paper/2303.08127/) (2023)
- [XWikiGen: Cross-lingual Summarization for Encyclopedic Text Generation in Low Resource Languages](https://mlsanity.com/paper/2303.12308/) (2023)
- [Impact of Model Size on Fine-tuned LLM Performance in Data-to-Text Generation: A State-of-the-Art Investigation](https://mlsanity.com/paper/2407.14088/) (2024)
- [GGP: A Graph-based Grouping Planner for Explicit Control of Long Text Generation](https://mlsanity.com/paper/2108.07998/) (2021)
- [GoalNet: Inferring Conjunctive Goal Predicates from Human Plan Demonstrations for Robot Instruction Following](https://mlsanity.com/paper/2205.07081/) (2022)
- [Parallel Hierarchical Transformer with Attention Alignment for Abstractive Multi-Document Summarization](https://mlsanity.com/paper/2208.07845/) (2022)
- [TrICy: Trigger-guided Data-to-text Generation with Intent aware Attention-Copy](https://mlsanity.com/paper/2402.01714/) (2024)

## Abstract (arXiv)

We propose an end-to-end, domain-independent neural encoder-aligner-decoder model for selective generation, i.e., the joint task of content selection and surface realization. Our model first encodes a full set of over-determined database event records via an LSTM-based recurrent neural network, then utilizes a novel coarse-to-fine aligner to identify the small subset of salient records to talk about, and finally employs a decoder to generate free-form descriptions of the aligned, selected records. Our model achieves the best selection and generation results reported to-date (with 59% relative improvement in generation) on the benchmark WeatherGov dataset, despite using no specialized features or linguistic resources. Using an improved k-nearest neighbor beam filter helps further. We also perform a series of ablations and visualizations to elucidate the contributions of our key model components. Lastly, we evaluate the generalizability of our model on the RoboCup dataset, and get results that are competitive with or better than the state-of-the-art, despite being severely data-starved.