Using reinforcement learning to autonomously identify sources of error for agents in group missions

When agents swarm to execute a mission, some of them frequently exhibit sudden failure, as observed from the command base. It is generally difficult to determine whether a failure is caused by actuators (hypothesis, ha) or sensors (hypothesis, hs) by solely relying on the communication between the command base and concerning agent. However, by instigating collusion between the agents, the cause of failure can be identified; in other words, we expect to detect corresponding displacements for ha but not for hs. In this study, we considered the question as to whether artificial intelligence can autonomously generate an action plan boldsymbolg to pinpoint the cause as aforedescribed. Because the expected response to boldsymbolg generally depends upon the adopted hypothesis [let the difference be denoted by D(boldsymbolg)], a formulation that uses Dleft(boldsymbolgright) to pinpoint the cause can be made. Although a boldsymbolg^* that maximizes D(boldsymbolg) would be a suitable action plan for this task, such an optimization is difficult to achieve using the conventional gradient method, as D(boldsymbolg) becomes nonzero in rare events such as collisions with other agents, and most swarm actions boldsymbolg give D(boldsymbolg)=0. In other words, throughout almost the entire space of boldsymbolg, D(boldsymbolg) has zero gradient, and the gradient method is not applicable. To overcome this problem, we formulated an action plan using Q-table reinforcement learning. Surprisingly, the optimal action plan generated via reinforcement learning presented a human-like solution to pinpoint the problem by colliding other agents with the failed agent. Using this simple prototype, we demonstrated the potential of applying Q-table reinforcement learning methods to plan autonomous actions to pinpoint the causes of failure.