# A Domain-Independent Agent Architecture for Adaptive Operation in Evolving Open Worlds

arXiv: https://arxiv.org/abs/2306.06272 · HTML: https://mlsanity.com/paper/2306.06272/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 71 references that are in the map.
- First posted on arXiv 9 Jun 2023.

## Builds on (most cited first, 10 of 10)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Selective Experience Replay for Lifelong Learning](https://mlsanity.com/paper/1802.10269/) (2018)
- [The Familiarity Hypothesis: Explaining the Behavior of Deep Open Set Methods](https://mlsanity.com/paper/2203.02486/) (2022)
- [AI Autonomy : Self-Initiated Open-World Continual Learning and Adaptation](https://mlsanity.com/paper/2203.08994/) (2022)
- [Efficient, Safe, and Probably Approximately Complete Learning of Action Models](https://mlsanity.com/paper/1705.08961/) (2017)
- [Safe Learning of Lifted Action Models](https://mlsanity.com/paper/2107.04169/) (2021)
- [On Exploiting Hitting Sets for Model Reconciliation](https://mlsanity.com/paper/2012.09274/) (2020)
- [COLIN: Planning with Continuous Linear Numeric Change](https://mlsanity.com/paper/1401.5857/) (2014)
- [Differential Assessment of Black-Box AI Agents](https://mlsanity.com/paper/2203.13236/) (2022)
- [Heuristic Search For Physics-Based Problems: Angry Birds in PDDL+](https://mlsanity.com/paper/2303.16967/) (2023)

## Built on by (most cited first, 3 of 3)

- [Epistemic Exploration for Generalizable Planning and Learning in Non-Stationary Settings](https://mlsanity.com/paper/2402.08145/) (2024)
- [Novelty Accommodating Multi-Agent Planning in High Fidelity Simulated Open World](https://mlsanity.com/paper/2306.12654/) (2023)
- [Requirements for Recognition and Rapid Response to Unfamiliar Events Outside of Agent Design Scope](https://mlsanity.com/paper/2504.12497/) (2025)

## Abstract (arXiv)

Model-based reasoning agents are ill-equipped to act in novel situations in which their model of the environment no longer sufficiently represents the world. We propose HYDRA - a framework for designing model-based agents operating in mixed discrete-continuous worlds, that can autonomously detect when the environment has evolved from its canonical setup, understand how it has evolved, and adapt the agents' models to perform effectively. HYDRA is based upon PDDL+, a rich modeling language for planning in mixed, discrete-continuous environments. It augments the planning module with visual reasoning, task selection, and action execution modules for closed-loop interaction with complex environments. HYDRA implements a novel meta-reasoning process that enables the agent to monitor its own behavior from a variety of aspects. The process employs a diverse set of computational methods to maintain expectations about the agent's own behavior in an environment. Divergences from those expectations are useful in detecting when the environment has evolved and identifying opportunities to adapt the underlying models. HYDRA builds upon ideas from diagnosis and repair and uses a heuristics-guided search over model changes such that they become competent in novel conditions. The HYDRA framework has been used to implement novelty-aware agents for three diverse domains - CartPole++ (a higher dimension variant of a classic control problem), Science Birds (an IJCAI competition problem), and PogoStick (a specific problem domain in Minecraft). We report empirical observations from these domains to demonstrate the efficacy of various components in the novelty meta-reasoning process.