# Reinforcement Learning for Few-Shot Text Generation Adaptation

arXiv: https://arxiv.org/abs/2111.11030 · HTML: https://mlsanity.com/paper/2111.11030/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 55 references that are in the map.
- First posted on arXiv 22 Nov 2021.

## Builds on (most cited first, 30 of 39)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables](https://mlsanity.com/paper/1903.08254/) (2019)
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [Automatic Detection of Generated Text is Easiest when Humans are Fooled](https://mlsanity.com/paper/1911.00650/) (2019)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Bayesian Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.03836/) (2018)
- [Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning](https://mlsanity.com/paper/2003.04390/) (2020)
- [A Survey of Reinforcement Learning Informed by Natural Language](https://mlsanity.com/paper/1906.03926/) (2019)
- [Meta-Reinforcement Learning of Structured Exploration Strategies](https://mlsanity.com/paper/1802.07245/) (2018)
- [Personalizing Dialogue Agents via Meta-Learning](https://mlsanity.com/paper/1905.10033/) (2019)
- [Induction Networks for Few-Shot Text Classification](https://mlsanity.com/paper/1902.10482/) (2019)
- [Meta-Q-Learning](https://mlsanity.com/paper/1910.00125/) (2019)
- [Jointly Measuring Diversity and Quality in Text Generation Models](https://mlsanity.com/paper/1904.03971/) (2019)
- [Toward Diverse Text Generation with Inverse Reinforcement Learning](https://mlsanity.com/paper/1804.11258/) (2018)
- [Domain Adaptive Dialog Generation via Meta Learning](https://mlsanity.com/paper/1906.03520/) (2019)
- [Survey on reinforcement learning for language processing](https://mlsanity.com/paper/2104.05565/) (2021)
- [Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning](https://mlsanity.com/paper/2010.01062/) (2020)
- [Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translation](https://mlsanity.com/paper/2103.02262/) (2021)
- [On Training Instance Selection for Few-Shot Neural Text Generation](https://mlsanity.com/paper/2107.03176/) (2021)
- [Meta-Learning for Low-resource Natural Language Generation in Task-oriented Dialogue Systems](https://mlsanity.com/paper/1905.05644/) (2019)
- [Graph-Evolving Meta-Learning for Low-Resource Medical Dialogue Generation](https://mlsanity.com/paper/2012.11988/) (2020)
- [MetaMT,a MetaLearning Method Leveraging Multiple Domain Data for Low Resource Machine Translation](https://mlsanity.com/paper/1912.05467/) (2019)

## Abstract (arXiv)

Controlling the generative model to adapt a new domain with limited samples is a difficult challenge and it is receiving increasing attention. Recently, methods based on meta-learning have shown promising results for few-shot domain adaptation. However, meta-learning-based methods usually suffer from the problem of overfitting, which results in a lack of diversity in the generated texts. To avoid this problem, in this study, a novel framework based on reinforcement learning (RL) is proposed. In this framework, to increase the sample utilization of RL and decrease its sample requirement, maximum likelihood estimation learning is incorporated into the RL process. When there are only a few in-domain samples available, experimental results on five target domains in two few-shot configurations show that this framework performs better than baselines.