mlsanityData to 14 Oct 2026

Value Under Ignorance in Universal Artificial Intelligence

arXiv 2512.17086 ↗First posted 18 Dec 2025cs.AIcited by 3

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We generalize the AIXI reinforcement learning agent to admit a wider class of utility functions. Assigning a utility to each possible interaction history forces us to confront the ambiguity that some hypotheses in the agent's belief distribution only predict a finite prefix of the history, which is sometimes interpreted as implying a chance of death equal to a quantity called the semimeasure loss. This death interpretation suggests one way to assign utilities to such history prefixes. We argue that it is as natural to view the belief distributions as imprecise probability distributions, with the semimeasure loss as total ignorance. This motivates us to consider the consequences of computing expected utilities with Choquet integrals from imprecise probability theory, including an investigation of their computability level. We recover the standard recursive value function as a special case. However, our most general expected utilities under the death interpretation cannot be characterized as such Choquet integrals.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 5 of its 22 references that are in the map. First posted on arXiv 18 Dec 2025.

Builds on

5 of 22 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Regret Bounds for Robust Online Decision Making9 Apr

Other references · 4

  1. 2022Settling the Reward Hypothesiscited by 33
  2. 2025Thinking Machines: A Survey of LLM based Reasoning Strategiescited by 25
  3. 2024Does RLHF Scale? Exploring the Impacts From Data, Model, and Methodcited by 23
  4. 2016Death and Suicide in Universal Artificial Intelligencecited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Solomonoff inductioncited by 2
  2. 2026Golden Handcuffs make safer AI agentscited by 1
  3. 2026Understanding Generalization Requires Universal Inductioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Generalised Bellman recurrence and three dualities in sequential decision-making
  2. 2026Rationality Measurement and Theory for Reinforcement Learning Agents
  3. 2025When a Reinforcement Learning Agent Encounters Unknown Unknowns
  4. 2024Partial Identifiability in Inverse Reinforcement Learning For Agents With Non-Exponential Discounting
  5. 2024Learning Utilities from Demonstrations in Markov Decision Processes
  6. 2026What Capable Agents Must Know: Selection Theorems for Robust Decision-Making under Uncertainty
  7. 2024Value of Information and Reward Specification in Active Inference and POMDPs
  8. 2024Bellman Unbiasedness: Toward Provably Efficient Distributional Reinforcement Learning with General Value Function Approximation