mlsanityData to 14 Oct 2026

Deep Extrapolation for Attribute-Enhanced Generation

arXiv 2107.02968 ↗First posted 7 Jul 2021cs.LG cs.CL q-bio.QMcited by 18

Not on the map. Why?

Abstract (arXiv)

Attribute extrapolation in sample generation is challenging for deep neural networks operating beyond the training distribution. We formulate a new task for extrapolation in sequence generation, focusing on natural language and proteins, and propose GENhance, a generative framework that enhances attributes through a learned latent space. Trained on movie reviews and a computed protein stability dataset, GENhance can generate strongly-positive text reviews and highly stable protein sequences without being exposed to similar data during training. We release our benchmark tasks and models to contribute to the study of generative modeling extrapolation and data-driven design in biology and chemistry.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 31 of its 53 references that are in the map. First posted on arXiv 7 Jul 2021.

Builds on

31 of 53 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2013Auto-Encoding Variational Bayescited by 4,053
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  5. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  6. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
  7. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
  8. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  9. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
  10. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  11. 2018MolGAN: An implicit generative model for small molecular graphscited by 393
  12. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 378
19 more
  1. 2017Wasserstein Auto-Encoderscited by 373
  2. 2017Toward Controlled Generation of Textcited by 272
  3. 2017Style Transfer from Non-Parallel Text by Cross-Alignmentcited by 248
  4. 2020How Neural Networks Extrapolate: From Feedforward to Graph Neural Networkscited by 222
  5. 2017Generalization and Equilibrium in Generative Adversarial Nets (GANs)cited by 166
  6. 2021Delving into Deep Imbalanced Regressioncited by 158
  7. 2017Controlling Linguistic Style Aspects in Neural Language Generationcited by 149
  8. 2020ProGen: Language Modeling for Protein Generationcited by 128
  9. 2019Conditioning by adaptive sampling for robust designcited by 111
  10. 2019Model Inversion Networks for Model-Based Optimizationcited by 83
  11. 2016Controlling Output Length in Neural Encoder-Decoderscited by 75
  12. 2018Unsupervised Text Style Transfer using Language Models as Discriminatorscited by 70
  13. 2020Autofocused oracles for model-based designcited by 50
  14. 2020Optimizing Molecules using Efficient Queries from Property Evaluationscited by 50
  15. 2020CoCon: A Self-Supervised Approach for Controlled Text Generationcited by 47
  16. 2021Protein sequence design with deep generative modelscited by 35
  17. 2018Design by adaptive samplingcited by 32
  18. 2021Adaptive machine learning for protein engineeringcited by 13
  19. 2019A Stable Variational Autoencoder for Text Modellingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 7; 2024: 3; 2025: 4; 2026: 2.

18 papers.

Sort:

By area:

  1. 2022Gradient-based Bi-level Optimization for Deep Learning: A Surveycited by 26
  2. 2023Bootstrapped Training of Score-Conditioned Generator for Offline Design of Biological Sequencescited by 23
  3. 2023Deep Generative Models for Detector Signature Simulation: A Taxonomic Reviewcited by 18
  4. 2022Plug & Play Directed Evolution of Proteins with Gradient-based Discrete MCMCcited by 17
  5. 2025Automating Steering for Safe Multimodal Large Language Modelscited by 17
  6. 2024Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Taskscited by 10
  7. 2024Robust Guided Diffusion for Offline Black-Box Optimizationcited by 8
  8. 2023Extrapolative Controlled Sequence Generation via Iterative Refinementcited by 7
  9. 2023Contrastive losses as generalized models of global epistasiscited by 2
  10. 2023Robust Model-Based Optimization for Challenging Fitness Landscapescited by 2
  11. 2023TemperatureGAN: Generative Modeling of Regional Atmospheric Temperaturescited by 1
  12. 2024Graph Community Augmentation with GMM-based Modeling in Latent Spacecited by 1
6 more
  1. 2025AffinityFlow: Guided Flows for Antibody Affinity Maturationcited by 1
  2. 2023Is Self-Supervised Pretraining Good for Extrapolation in Molecular Property Prediction?cited by 0
  3. 2025Towards Lightweight, Adaptive and Attribute-Aware Multi-Aspect Controllable Text Generation with Large Language Modelscited by 0
  4. 2025Learning Extrapolative Sequence Transformations from Markov Chainscited by 0
  5. 2026An Information Theoretic Framework for Graph Novelty Generation via Latent Mixture Modelingcited by 0
  6. 2026Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Modelscited by 0