Skip to content
← All notes

Human and AI teams: who should make the final decision?

The useful boundary is not how difficult the decision is. It is whether the decision is reversible, and who has to answer for it if it is wrong.

Leadership7 min read

Most attempts to draw this line sort decisions by complexity, giving the simple ones to the machine and keeping the hard ones. It sounds sensible and it fails immediately, because plenty of simple decisions are catastrophic and plenty of complex ones are trivially reversible.

Two better questions: can this be undone, and who answers for it if it is wrong.

Reversibility is the first test

A decision you can undo cheaply is a decision you can afford to be wrong about, which means it can be delegated and corrected. A decision you cannot undo has to be right the first time, and that is where a human belongs regardless of how easy it looks.

Sending a message, publishing something, deleting data, committing money, telling somebody they did not get the job: all irreversible, several of them trivial to make. Difficulty is not the variable.

Decisions sorted by whether they can be undone rather than by how hard they are. Several of the easiest ones, sending, publishing, deleting, committing money, cannot be taken back at all.
Difficulty is not the variable. Reversibility is.

Accountability is the second, and it does not move

Wherever a decision is made, somebody is answerable for it. That never transfers to a system, whatever the workflow diagram says.

This gives a clean rule: if you would not be comfortable defending this outcome, you cannot delegate the decision to something that will not be there to defend it. The comfort has to come before the delegation rather than after the failure.

A workable division

Applying both tests produces something you can actually operate rather than a principle you agree with and ignore.

  • Machine decides, nobody reviews: reversible, low consequence, high volume. Sorting, routing, drafting, first-pass formatting.
  • Machine recommends, human decides: consequential but not irreversible, and the human must be able to evaluate the recommendation. Prioritisation, resourcing, most analysis.
  • Human decides, machine assists with information only: anything irreversible, anything about a specific person, anything where being wrong is expensive or public.
  • Human decides, machine stays out: anything where the reasoning has to be defensible to somebody affected by it. Hiring, dismissal, discipline, safety.

The failure mode nobody names

The dangerous pattern is not a machine deciding something it should not. It is a human formally deciding while having no capacity to evaluate what they are approving.

This is rubber-stamping with a name on it, and it is worse than automation because it produces the appearance of oversight without any. If the reviewer cannot say what would make them reject the recommendation, they are not reviewing it, and the accountability sitting on them is fiction.

So the real requirement for any human-in-the-loop step is that the human is genuinely able to evaluate the thing. If they are not, either give them what they need, or stop pretending the step is a control.

An approval step where the approver has no capacity to evaluate what they are signing. The oversight is formally present and functionally absent, and the accountability resting on them is fiction.
If they cannot say what would make them reject it, they are not reviewing it.

Common questions

When should a human make the final decision instead of AI?
When the decision is hard to reverse, when it concerns a specific person, or when somebody would have to defend the reasoning to whoever is affected. Reversibility and accountability are better tests than difficulty, because plenty of easy decisions are irreversible.
What is human-in-the-loop and when does it fail?
It is a required human approval step before an automated decision takes effect. It fails when the human cannot actually evaluate what they are approving, which produces the appearance of oversight with none of the substance, and puts accountability on somebody with no ability to exercise it.
Who is accountable when an AI system makes a bad decision?
The people who deployed it and the person who owned the outcome. Accountability does not transfer to a system, whatever the workflow says. That is why deciding where a human must sit is a governance question and not a technical one.

PDP Quest exists because of the problem underneath all of these: when output stops indicating capability, you need another way to know who can actually do the work.

See how verification works →