Adaptive Oversight Subversion

Tue, 13 Oct 2026 00:00:00 +0000

A model which works on a long task might take actions which nobody sanctioned. For example, in a cyber-security task it might forget that a system is out of scope, or deliberately attack it, because it became frustrated when it could not solve the task. A case of this kind is the that the UK AI Security Institute reported during its cyber testing. To prevent that, a chain-of-thought (CoT) monitor reads the reasoning and the tool calls of the model at every step (inspired by Baker et al., ), gives the step a suspiciousness score (inspired by Greenblatt et al., ), and blocks it when the score is too high. However, a blocked model still has a task to solve. So this project studies how models react to a block, whether they try to evade the monitor, and which strategies they adopt.

AI Safety | Marcello Maugeri

Adaptive Oversight Subversion