mlsanityData to 7 Oct 2026

Does the Order of Training Samples Matter? Improving Neural Data-to-Text Generation with Curriculum Learning

arXiv 2102.03554 ↗First posted 6 Feb 2021cs.CLcited by 34

Not on the map. Why?

Abstract (arXiv)

Recent advancements in data-to-text generation largely take on the form of neural end-to-end systems. Efforts have been dedicated to improving text generation systems by changing the order of training samples in a process known as curriculum learning. Past research on sequence-to-sequence learning showed that curriculum learning helps to improve both the performance and convergence speed. In this work, we delve into the same idea surrounding the training samples consisting of structured data and text pairs, where at each update, the curriculum framework selects training samples based on the model's competence. Specifically, we experiment with various difficulty metrics and put forward a soft edit distance metric for ranking training samples. Our benchmarks show faster convergence speed where training time is reduced by 38.7% and performance is boosted by 4.84 BLEU.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 15 of its 33 references that are in the map. First posted on arXiv 6 Feb 2021.

Builds on

15 of 33 references.

References · 15

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 267
  3. 2017The E2E Dataset: New Challenges For End-to-End Generationcited by 252
  4. 2017Challenges in Data-to-Document Generationcited by 226
  5. 2019Competence-based Curriculum Learning for Neural Machine Translationcited by 201
  6. 2019Levenshtein Transformercited by 197
  7. 2017Curriculum Learning and Minibatch Bucketing in Neural Machine Translationcited by 63
  8. 2020Norm-Based Curriculum Learning for Neural Machine Translationcited by 60
  9. 2019Curriculum Learning for Domain Adaptation in Neural Machine Translationcited by 59
  10. 2018An Empirical Exploration of Curriculum Learning for Neural Machine Translationcited by 58
  11. 2020Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondencecited by 25
  12. 2020DART: A Lightweight Quality-Suggestive Data-to-Text Annotation Toolcited by 12
3 more
  1. 2020Unsupervised Pidgin Text Generation By Pivoting English Data and Self-Trainingcited by 9
  2. 2017Neobility at SemEval-2017 Task 1: An Attention-based Sentence Similarity Modelcited by 3
  3. 2020Safe Handover in Mixed-Initiative Control for Cyber-Physical Systemscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 6; 2023: 4; 2024: 9; 2025: 6; 2026: 5.

34 papers.

Sort:

By area:

  1. 2025Principled Data Selection for Alignment: The Hidden Risks of Difficult Examplescited by 37
  2. 2024LeanAgent: Lifelong Learning for Formal Theorem Provingcited by 34
  3. 2025RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Trainingcited by 31
  4. 2021Deep Spatially and Temporally Aware Similarity Computation for Road Network Constrained Trajectoriescited by 29
  5. 2024Strategic Data Ordering: Enhancing Large Language Model Performance through Curriculum Learningcited by 28
  6. 2024Denoising Task Difficulty-based Curriculum for Training Diffusion Modelscited by 26
  7. 2021On Training Instance Selection for Few-Shot Neural Text Generationcited by 25
  8. 2022An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Modelscited by 20
  9. 2021Time-Aware Ancient Chinese Text Translation and Inferencecited by 10
  10. 2025Data Efficacy for Language Model Trainingcited by 8
  11. 2023Too Much Information: Keeping Training Simple for BabyLMscited by 6
  12. 2024HiCuLR: Hierarchical Curriculum Learning for Rhetorical Role Labeling of Legal Documentscited by 6
22 more
  1. 2021The SelectGen Challenge: Finding the Best Training Samples for Few-Shot Neural Text Generationcited by 5
  2. 2022Decorate the Examples: A Simple Method of Prompt Design for Biomedical Relation Extractioncited by 5
  3. 2024LLM Attributor: Interactive Visual Attribution for LLM Generationcited by 5
  4. 2024Improving Language Models Trained on Translated Data with Continual Pre-Training and Dictionary Learning Analysiscited by 5
  5. 2022In-sample Curriculum Learning by Sequence Completion for Natural Language Generationcited by 4
  6. 2023Curriculum-guided Abstractive Summarization for Mental Health Online Postscited by 3
  7. 2025GIT-CXR: End-to-End Transformer for Chest X-Ray Report Generationcited by 3
  8. 2025Task-Informed Anti-Curriculum by Masking Improves Downstream Performance on Textcited by 3
  9. 2022Adaptive Multi-Corpora Language Model Training for Speech Recognitioncited by 2
  10. 2026A Severity-Based Curriculum Learning Strategy for Arabic Medical Text Generationcited by 2
  11. 2026Demystifying Data Organization for Enhanced LLM Trainingcited by 2
  12. 2022Medical Scientific Table-to-Text Generation with Human-in-the-Loop under the Data Sparsity Constraintcited by 1
  13. 2022High-Resource Methodological Bias in Low-Resource Investigationscited by 1
  14. 2024Data Debugging is NP-hard for Classifiers Trained with SGDcited by 1
  15. 2024Curriculum Learning for Cross-Lingual Data-to-Text Generation With Noisy Datacited by 1
  16. 2024Learning from Impairment: Leveraging Insights from Clinical Linguistics in Language Modelling Researchcited by 1
  17. 2025CuDIP: Enhancing Theorem Proving in LLMs via Curriculum Learning-based Direct Preference Optimizationcited by 1
  18. 2026CLewR: Curriculum Learning with Restarts for Machine Translation Preference Learningcited by 1
  19. 2023Curriculum-Guided Abstractive Summarizationcited by 0
  20. 2023Towards Zero-Shot Personalized Table-to-Text Generation with Contrastive Persona Distillationcited by 0
  21. 2026Severity-Aware Curriculum Learning with Multi-Model Response Selection for Medical Text Generationcited by 0
  22. 2026Disentangling Curriculum Learning in NLP: Towards a Unifying Taxonomycited by 0