mlsanityData to 14 Oct 2026

OMNI: Open-endedness via Models of human Notions of Interestingness

arXiv 2306.01711 ↗First posted 2 Jun 2023cs.AI cs.LGcited by 70

Not on the map. Why?

Abstract (arXiv)

Open-ended algorithms aim to learn new, interesting behaviors forever. That requires a vast environment search space, but there are thus infinitely many possible tasks. Even after filtering for tasks the current agent can learn (i.e., learning progress), countless learnable yet uninteresting tasks remain (e.g., minor variations of previously learned tasks). An Achilles Heel of open-endedness research is the inability to quantify (and thus prioritize) tasks that are not just learnable, but also textitinteresting (e.g., worthwhile and novel). We propose solving this problem by textitOpen-endedness via Models of human Notions of Interestingness (OMNI). The insight is that we can utilize foundation models (FMs) as a model of interestingness (MoI), because they textitalready internalize human concepts of interestingness from training on vast amounts of human-generated data, where humans naturally write about what they find interesting or boring. We show that FM-based MoIs improve open-ended learning by focusing on tasks that are both learnable textitand interesting, outperforming baselines based on uniform task sampling or learning progress alone. This approach has the potential to dramatically advance the ability to intelligently select which tasks to focus on next (i.e., auto-curricula), and could be seen as AI selecting its own next task to learn, facilitating self-improving AI and AI-Generating Algorithms. Project website at https://www.jennyzhangzt.com/omni/

Key facts. Cited by 70 papers in the map (data through 14 Oct 2026). Builds on 62 of its 79 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

62 of 79 references.

References · 62

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  9. 2017Deep reinforcement learning from human preferencescited by 3,912
  10. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  11. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  12. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
38 more
  1. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  2. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  3. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  5. 2019Solving Rubik's Cube with a Robot Handcited by 734
  6. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  7. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  8. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
  9. 2015Illuminating search spaces by mapping elitescited by 425
  10. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  11. 2023Reward Design with Language Modelscited by 261
  12. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  13. 2017Automated Curriculum Learning for Neural Networkscited by 257
  14. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 224
  15. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 219
  16. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  17. 2017Teacher-Student Curriculum Learningcited by 210
  18. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 209
  19. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
  20. 2020Prioritized Level Replaycited by 198
  21. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 192
  22. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 187
  23. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  24. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164
  25. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  26. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  27. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 106
  28. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  29. 2021Skill Induction and Planning with Latent Languagecited by 99
  30. 2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 88
  31. 2020Automatic Curriculum Learning through Value Disagreementcited by 87
  32. 2019AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligencecited by 86
  33. 2020Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Surveycited by 85
  34. 2019A survey on intrinsic motivation in reinforcement learningcited by 81
  35. 2018The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communitiescited by 78
  36. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 72
  37. 2020Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Explorationcited by 68
  38. 2013Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robotscited by 58

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 15; 2025: 25; 2026: 22.

70 papers.

Sort:

By area:

  1. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 551
  2. 2024Automated Design of Agentic Systemscited by 407
  3. 2023GenSim: Generating Robotic Simulation Tasks via Large Language Modelscited by 125
  4. 2024A Survey on Large Language Model-Based Game Agentscited by 112
  5. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 102
  6. 2025STP: Self-play LLM Theorem Provers with Iterative Conjecturing and Provingcited by 79
  7. 2025A Self-Improving Coding Agentcited by 79
  8. 2023Quality-Diversity through AI Feedbackcited by 40
  9. 2026Group-Evolving Agents: Open-Ended Self-Improvement via Experience Sharingcited by 36
  10. 2024Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Modelscited by 35
  11. 2024Data-driven Discovery with Large Generative Modelscited by 34
  12. 2024Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Modelscited by 33
38 more
  1. 2025AutoDiscovery: Open-ended Scientific Discovery via Bayesian Surprisecited by 33
  2. 2025SIMA 2: A Generalist Embodied Agent for Virtual Worldscited by 31
  3. 2024DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedbackcited by 25
  4. 2025Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesiscited by 22
  5. 2024Automating the Search for Artificial Life with Foundation Modelscited by 20
  6. 2025Truly Self-Improving Agents Require Intrinsic Metacognitive Learningcited by 15
  7. 2023LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teacherscited by 14
  8. 2026Agentic Artificial Intelligence (AI): Architectures, Taxonomies, and Evaluation of Large Language Model Agentscited by 14
  9. 2025Automated Skill Discovery for Language Agents through Exploration and Iterative Feedbackcited by 12
  10. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 12
  11. 2025Automated Capability Discovery via Foundation Model Self-Explorationcited by 10
  12. 2025Bootstrapping Task Spaces for Self-Improvementcited by 10
  13. 2024Stop Wandering, Find the Keys: LLMs Discriminate Key States for Efficient Multi-Agent Explorationcited by 9
  14. 2025SENSEI: Semantic Exploration Guided by Foundation Models to Learn Versatile World Modelscited by 9
  15. 2024From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Modelscited by 8
  16. 2024Automating Traffic Model Enhancement with AI Research Agentcited by 8
  17. 2025Intrinsically-Motivated Humans and Agents in Open-World Explorationcited by 8
  18. 2024Bayesian Statistical Modeling with Predictors from LLMscited by 7
  19. 2025MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spacescited by 7
  20. 2025SPARQ: Synthetic Problem Generation for Reasoning via Quality-Diversity Algorithmscited by 7
  21. 2026Digital Red Queen: Adversarial Program Evolution in Core War with LLMscited by 7
  22. 2023ACES: Generating Diverse Programming Puzzles with with Autotelic Generative Modelscited by 6
  23. 2025LLM-First Search: Self-Guided Exploration of the Solution Spacecited by 6
  24. 2025Foundation Model Self-Play: Open-Ended Strategy Innovation via Foundation Modelscited by 6
  25. 2026Toward an automated science of the mindcited by 5
  26. 2024Automated Rewards via LLM-Generated Progress Functionscited by 4
  27. 2025Safety Must Precede the Deployment of Open-Ended AIcited by 3
  28. 2026Interestingness as an Inductive Heuristic for Future Compression Progresscited by 3
  29. 2023Learning Curricula in Open-Ended Worldscited by 2
  30. 2024Syllabus: Portable Curricula for Reinforcement Learning Agentscited by 2
  31. 2025Shall We Play a Game? Language Models for Open-ended Wargamescited by 2
  32. 2026Heuresis: Search Strategies for Autonomous AI Research Agents Across Quality, Diversity and Noveltycited by 2
  33. 2026DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculumcited by 2
  34. 2023Quality Diversity in the Amorphous Fortress (QD-AF): Evolving for Complexity in 0-Player Gamescited by 1
  35. 2024Evolutionary Computation for the Design and Enrichment of General-Purpose Artificial Intelligence Systems: Survey and Prospectscited by 1
  36. 2025Expedition & Expansion: Leveraging Semantic Representations for Goal-Directed Exploration in Continuous Cellular Automatacited by 1
  37. 2025Guiding Evolution of Artificial Life Using Vision-Language Modelscited by 1
  38. 2025AutoContext: Instance-Level Context Learning for LLM Agentscited by 1

50 most cited of 70 All 70 (JSON)