Behavioural self-monitoring for autonomous earth observation agents: The attribute reliability monitor (ARM)

Language and vision language models can support autonomous systems, but changing operating conditions may compromise the reliability of their semantic outputs. This work introduces the Attribute Reliability Monitor (ARM), a framework that accumulates evidence of suspicion over time to assess individual semantic attributes. ARM accommodates different detectors and temporal monitoring methods, while leaving intervention policies to the consuming system. An initial implementation was evaluated using a specialized vision language model developed for satellite thermal infrared imagery. Monitoring followed a black box approach, using only observable responses without access to internal model states, confidence scores, or external reference answers. Under controlled input degradation, disagreement between responses provided evidence for selective suspension of attributes. The results illustrate how external behavioral monitoring can inform continued use of model outputs, while distinguishing operational suspicion from semantic correctness.