Human-in-the-Loop Is a Stage, Not a Policy

Part 4 of the Managing Agents Like People series. New to the series? Start with Part 1: Managing Agents Is Like Managing People.

"Keep a Human in the Loop" Answers the Wrong Question

Most AI usage policies that companies write in their first year contain some version of the same sentence: a human must remain in the loop for any action an agent takes. It reads well. It signals caution, it's easy to audit, and it gives everyone a clear answer when someone asks whether the new automation is safe.

The trouble is what the sentence leaves out. It doesn't say which tasks, which loop, or for how long. Written as a blanket, permanent rule, it's the governance equivalent of requiring manager sign-off on every action an employee takes, for their entire tenure, regardless of track record. No company runs its people that way past the first few weeks of a new hire. Yet that's exactly the policy many organizations have quietly adopted for their agents, and then never revisited.

This isn't an argument against human oversight. Human review matters, and for some tasks it should never go away. It's an argument that "human in the loop, yes or no" was never the useful question. Even the EU's AI Act, which makes human oversight a legal requirement for high-risk AI systems, doesn't treat it as one fixed setting: Article 14 says the oversight measures "shall be commensurate with the risks, level of autonomy and context of use." The useful questions are which loop a human should be in for a given task, and what would have to be true for that to change.

Nobody Manages a Senior Employee Like a First-Week Hire

Think about how a reasonable manager handles a new accounts payable clerk. In the first week, every invoice they process gets looked at before it's paid. Nobody calls that micromanagement. The person is new, the system is new to them, and close review is how both sides learn where the gaps are.

By month three, that same manager is spot-checking a sample and reviewing the exceptions report. By the end of the year, the clerk is handling routine invoices on their own and bringing the strange ones forward. The oversight didn't disappear. It changed shape, because the evidence justified the change.

In the language of the Situational Leadership model, which I walked through in The Situational Leadership Curve, Applied to Agents, the new clerk starts in a directing stage and works toward delegating, task by task. The point I want to pull from it here is narrower: the directing stage was always meant to be temporary. The failure isn't putting a new agent in that stage. The failure is leaving it frozen there long after the evidence says otherwise. The costs of that, from throughput collapse to reviewers who stop really reading what they approve, were the subject of The Agentic Micro-Manager: What Gets Lost When You Approve Every Step. This post is about the policy design that avoids it.

There Are Three Loops, Not One

"The loop" sounds like a single thing. In practice, a human can sit in at least three very different places relative to an agent's work, and each one protects against different failures.

Approval on actions. The human signs off before the agent does something: before the vendor email goes out, before the CRM record is updated, before the code change is merged. This is the heaviest loop. It catches problems before they happen, and it's the right default for a task that's new or poorly understood. It's also the loop that scales worst, because every unit of work waits on a person.

Review of outcomes. The agent does the work, and a human checks the result: a sample of categorized support tickets, the weekly report before it's distributed, a reconciliation against the source system at month-end. This loop catches problems after they happen but before they compound. It's where most mature delegation lives, for people and agents alike, because it focuses attention on whether the result is right rather than on whether each step looked reasonable.

Escalation on exceptions. The agent handles the routine work and routes anything unusual to a person: an invoice that doesn't match its purchase order, a customer request outside known categories, a change that touches a system it wasn't scoped for. This loop only works if the escalation rules are explicit, because an agent won't reliably flag "this looks off" on general judgment the way an experienced person will.

These loops aren't mutually exclusive. A well-run task often uses two of them at once. But a policy that says "human in the loop" without naming which one usually defaults to the first, everywhere, indefinitely. That's the expensive choice, and it's rarely the one anyone would pick on purpose.

Write the Exit Criteria Before the Agent Starts

If approval on actions is a stage, it needs a defined way out. The most useful design move I know is borrowed directly from onboarding: agree on what "ready for more rope" looks like before day one, not after someone gets impatient.

For an agent drafting replies to routine support tickets, that might sound like: once it has handled a meaningful volume of tickets with no substantive corrections from reviewers, and has correctly escalated the ones outside its scope instead of guessing, approval moves from every reply to a daily sample. For an agent that updates CRM records from meeting notes, it might be a clean reconciliation over several consecutive weeks before it stops waiting for a sign-off on each record.

The specific thresholds are a judgment call for the people who own the work. What matters is that they're written down, tied to evidence rather than to the calendar or a general feeling that "it's been fine lately," and attached to the task, not the agent. The same agent can be ready for outcome review on one task and still need step approval on another.

Exit criteria should also run in reverse. A confident wrong answer, an action outside its scope, or a model or prompt change should move a task back a stage, at least until the evidence rebuilds. A stage you can only ever leave isn't a stage. It's a countdown.

Some Tasks Should Stay at Sign-Off Permanently, by Design

None of this means every task should graduate. Some should keep a human approval on every action indefinitely, and the honest version of this argument has to say so plainly.

Wire transfers are the clearest example from the human world. Most companies require a second signature on large payments no matter how senior or trusted the person requesting them is. That isn't a statement about the requester's competence. It's a statement about the action: irreversible, high value, and a common target for fraud. The cost of a rare miss outweighs years of smooth runs, so the control stays.

The same logic applies to agents. Moving money, deleting data that can't be restored, sending something to a regulator or a large customer list, changing access permissions in production: these are reasonable candidates for permanent approval on the action itself, however good the track record gets. The test is irreversibility and blast radius, not how comfortable anyone feels with the technology.

The difference between this and a blanket policy is that it's a decision. Someone looked at the task, named why it keeps sign-off, and wrote that down. That's very different from sign-off that persists simply because nobody ever asked whether it still made sense.

Graduating a Task Moves the Human, It Doesn't Remove Them

The worry behind blanket human-in-the-loop policies is legitimate: if humans step back, who answers for the result? The answer is the same people who answered for it before. Moving a task from action approval to outcome review doesn't transfer accountability to the agent. It changes how the accountable person exercises it.

In some ways, the later loops ask more of that person, not less. Approving individual steps can be done half-attentively. Reviewing outcomes well requires knowing what a correct result looks like, sampling with intent, and following up when something is off. Owning escalation rules requires thinking ahead about what "unusual" means for this task. All of that is real oversight, and arguably more meaningful oversight than clicking approve on a queue. It's also a different skill from doing the work: Microsoft researchers name "a shift in users' roles from production to evaluation" as one of the main reasons people lose productivity with generative AI, a pattern automation research has tracked for decades.

Keeping humans in the loop and letting agents earn more trust are often argued as opposing positions. They aren't. A task that graduates through stages, with written criteria and a path back down, is how you keep human judgment involved without making it the bottleneck. As I argued in Managing Agents Is Like Managing People, at some point you have to let go and trust the outcome. Staging is how you get there deliberately, with evidence, instead of either never letting go or letting go all at once.

A practical place to start: list the agent-driven tasks you run today, note which loop each one is in, and ask whether anyone could say what would have to be true for that to change. Where the answer is "nobody knows," that's a policy, not a stage. Put a review date on it.

Subscribe for more like this.

In the spirit of this site: this article was drafted with AI assistance and edited by Matt before publishing — the same kind of human-in-the-loop review this piece argues for.

About the Author: Matt Shirel writes Praxis CTO, exploring what real technology leadership looks like for growing companies that have outgrown ad hoc decision-making. He brings over 20 years in enterprise IT — spanning infrastructure architecture, cloud strategy, and technology budget ownership — and works hands-on in applied AI, building multi-agent and RAG workflows with tools like n8n and AI coding agents — which means deciding, task by task, which of his own agents still need a sign-off and which have earned a lighter touch.