Human in the loop — oversight as design

The short answer. Human in the loop means a person makes or confirms the consequential decision the AI informs — the model proposes, the clinician disposes. It is the most common risk control in clinical AI and the most commonly overstated one: oversight only works if the human genuinely can and does dissent, which is why override rates are its health check.

“A clinician reviews every output” is the sentence that moves most clinical AI down a risk tier. Sometimes it deserves to: a person with authority, time, and information genuinely deciding is a real control.

The governance caution is automation bias — humans defer to confident systems, and a reviewer who agrees with the model 100% of the time may be a rubber stamp wearing a control’s name. Two questions test whether the loop is real: can the human dissent (authority, time, enough context to disagree), and do they (a plausible, monitored override rate)? A human-in-the-loop claim that cannot survive those two questions should not be lowering anyone’s risk tier — and the claim itself belongs in writing, which is why the HAI-OS registry records each model’s human-oversight arrangement as a governed field rather than a slide-deck assurance.

Where do you actually stand? The free RUAIH readiness score maps your organisation against the five focus areas in about eight minutes, and the published crosswalk shows how each control lands across RUAIH, CHAI and the NIST AI RMF.

Human in the loop — oversight as design

Keep reading