Human in the loop: why AI agents should ask before they act

In July 2025, SaaStr founder Jason Lemkin was testing an AI coding agent on Replit when, by his account, it deleted his company's live production database in the middle of a code freeze. Fortune reported that the agent admitted running commands it hadn't been given permission to run, despite instructions to wait for human approval, and told him: "This was a catastrophic failure on my part." Replit's CEO called it "unacceptable and should never be possible," and the company announced automatic separation of development and production databases, better rollback, and a planning-only mode. The lesson isn't that agents are useless. It's that an instruction is not a control. Human in the loop AI is how you turn "please ask first" into something the software enforces, and this guide shows how to do it without strangling the agent.

What is human in the loop AI?

Human in the loop means a person reviews and approves an AI's decision before it takes effect. For an AI agent, that usually means the agent does the work up to the point of acting, then waits for a yes before it sends, pays, deletes or changes anything that matters. (In model training, the term also covers people labelling data and correcting answers; this guide is about agents at work.)

It's one of three postures you'll hear about:

Human in the loop: the agent proposes, a person approves each important action.
Human on the loop: the agent acts by itself while a person watches and can step in or stop it.
Human out of the loop: the agent acts alone, and people review results afterwards, if at all.

Researchers have broken this down further. A 2025 paper, "Levels of Autonomy for AI Agents," describes five levels by the role the person plays: operator (you drive, the agent helps when asked), collaborator (you plan and work together), consultant (the agent does most of the work and asks for your expertise and preferences), approver (the agent runs the job and comes to you only for consequential actions or when it's blocked) and observer (the agent does everything and you watch the logs, with only an emergency off switch).

For most businesses using agents today, approver is the sweet spot: the agent does the legwork, and you make the calls that count.

Why agents need it more than chatbots

A chatbot produces words. An agent produces actions. That changes the stakes.

Even words carry liability. In 2024 a Canadian tribunal ordered Air Canada to compensate a customer after its website chatbot gave him wrong information about a bereavement discount. The airline argued the chatbot was "a separate legal entity"; the tribunal said it was "still just a part of Air Canada's website." If a business answers for what its bot says, it certainly answers for what its agent does.

Three things make agents risky without a person in the loop:

They can be wrong with confidence. A model can misread a situation and then act on its misreading, step after step.
They can be manipulated. An agent that reads email or web pages can meet planted instructions, known as prompt injection. OWASP's example is an assistant allowed to read a mailbox that also has a send function: a malicious email tells it to find sensitive messages and forward them to an attacker. More in AI agent security and prompt injection.
They can have too much power. OWASP calls this excessive agency: too many tools, too broad permissions, or too much autonomy. Its recommended fix includes requiring "a human to approve high-impact actions before they are taken."

Not every action needs a human: sort by risk

If an agent asks about everything, people stop reading and start tapping "yes". The trick is to ask about the right things. OpenAI's guide suggests rating each tool low, medium or high risk by four questions: does it only read or also write, can it be undone, what permissions does it need, and does it touch money?

Sort actions by risk: the agent reads and drafts on its own; anything that can’t be undone waits for a person.

The most important of those is reversibility. Relabelling an email can be undone in a second. A sent email can't be unsent. A deleted database without a backup is gone. Here's a tiered policy built on that idea:

Tier	What the agent does	Examples	Default rule
0. Read	Looks things up	Read email, query a database, check a website	Runs freely, logged
1. Draft	Prepares something for a person	Draft replies, prepare an invoice, propose a config change	Runs freely; a person sends or applies
2. Reversible, internal	Changes something easy to undo, inside the business	Label emails, update a CRM field, file a receipt, create a task	Ask during a trial period, then allow with logging
3. External or hard to undo	Anything another person sees, or that can't be taken back	Send an email, message a customer, publish a post, change a customer's account	Ask every time, unless a standing rule covers exactly this kind of action
4. Money, deletion, access	Moves money, destroys data, changes who can get in	Payments, refunds, deleting records, changing passwords or permissions, production database changes	Always ask; consider a second approver; some actions never allowed

OpenAI's guide puts tier 4 in plain terms: actions that are "sensitive, irreversible, or have high stakes," such as cancelling orders, authorising large refunds or making payments, should trigger human oversight "until confidence in the agent's reliability grows."

Approvals that people actually read

An approval is only a control if the person reads it. Rubber-stamping has a name, automation bias: the tendency to over-rely on what an automated system recommends. Good approval requests are designed against it:

One sentence on what will change. "Move line 4471 to plan home-20M on edge-router-2", not "Approve action?"
What was checked, and what wasn't. "Everything before this only read" is reassuring; "I couldn't confirm the customer's address" is a reason to look closer.
Batched when it's the same decision. "Send the 3 drafts" or "Review first", rather than three separate pings.
Where you are. A button on your phone gets a faster, more considered answer than a tab you'll find tomorrow.
Standing permissions, earned. Once a kind of action has been right for weeks, let the agent go ahead on exactly that kind, and keep asking about everything else.

Big AI products have converged on this. ChatGPT agent, before OpenAI retired it in favour of ChatGPT Work and Dots, showed confirmation prompts before steps like submitting forms or sending emails, and let you interrupt, redirect or pause it at any time. When OpenAI's always-on Dots are idle, they research with read-only access to your apps, so they can't send or change anything; actions that touch accounts or share information go through an automatic review first. In one tester's case, a dot drafted an invoice and waited for approval before sending it.

Budgets: a limit on money and on steps

The second control is a budget. An agent works in a loop, and a loop that goes wrong can keep going. Anthropic recommends stopping conditions such as a maximum number of iterations. OpenAI suggests limits on retries, escalating to a person when an agent keeps failing, for example when it can't understand what a customer wants after several attempts.

Set budgets on two things:

Spend per job. A ceiling on what one job may cost in AI usage. When it's hit, the agent stops and asks instead of carrying on.
Steps or time per job. A cap that catches an agent going round in circles even when each step is cheap.

Audit trails: every step on record

The third control is memory you can check. An audit trail records, for every job:

what the agent read (emails, pages, records),
every tool it called, with the inputs and the result,
every approval asked for, who answered, and when,
what it cost.

It's how you find out why something happened, prove what was authorised, and improve the agent's instructions. OWASP notes that logging and monitoring don't prevent harm but limit it, by catching bad actions early. Judge an agent by its log, not only by its summary.

Least privilege: limit what it can reach

Approvals catch bad actions. Least privilege makes many of them impossible. OWASP's advice:

Give only the tools the job needs. A summarising agent gets a read-only mail tool, not one that can also send.
Use narrow permissions. Read-only database access for reports; the person's own scoped login rather than an all-powerful account.
Enforce permissions in the system, not the prompt. The database should refuse the delete, whatever the agent asks.
Separate test from production. Replit's own fix after the incident.
Keep secrets away from the model. Passwords and keys belong in a vault the agent's software uses on its behalf; the model itself never needs to see them.

A practical policy a business can adopt

Here's a starting policy you can copy, adapt and put in your team handbook.

Every agent has an owner, a named person responsible for what it does.
Agents start read-only. New jobs begin with reading, reporting and drafting.
Writes need approval by default. Anything outside tier 0 and 1 asks first until the owner grants a standing permission for that exact kind of action.
Money, deletion and access always need a person, and payments above a set amount need two.
Every job has a budget, in money and in steps. Hitting it means stop and ask.
Agents never see passwords. Credentials stay in a vault; access is the minimum the job needs.
Test and production are separate, and agents work in test unless the job is explicitly live.
Everything is logged and kept, and the owner reviews a sample every week.
Customer-facing bots don't act. A bot that talks to the public answers and hands over; it doesn't change accounts.
Anyone can hit stop. There's a clear way to pause any agent, and people know where it is.

What the law says

Most business agents aren't regulated as "high-risk" systems, but the principle is heading into law. The EU AI Act's Article 14 requires high-risk AI systems to be designed so that people can oversee them effectively while in use. The people overseeing them must be able to understand the system's limits, stay aware of automation bias, override or reverse its output, and interrupt it "through a 'stop' button or a similar procedure." That's a sound checklist for any agent, regulated or not.

How crew bot puts a human in the loop

crew bot is built around the approver model. It does the work, and asks first before it sends an email, changes an account or runs anything that writes, unless you've told it to go ahead. Approvals and budget questions arrive on your phone with buttons, so "yes" doesn't wait for you to get back to your desk.

Every job has a budget, fifty US cents by default: if a job needs more, it stops and asks. Logins are encrypted and never shown to a model; you type passwords into sites yourself, and SSH to your servers uses five-minute certificates. Every step is on record: each job keeps what it read, clicked, ran and found, so you can check its work. When it uses your computer, you watch every move live and can take over at any time.

And on your website, the support chat follows rule 9 above: because it talks to strangers, its bot has no tools at all. It answers from what your site and team say, and hands the visitor to a person when it should.

For the bigger picture, see what is an AI agent? and how AI agents work.

FAQ

What is an example of human in the loop AI?

An AI agent drafts replies to the morning's customer emails, then sends you a message: "3 drafts ready. Send, or review first?" Nothing goes out until you tap send. The agent did the reading and writing; you made the decision that affects customers.

What's the difference between human in the loop and human on the loop?

In the loop, the agent waits for a person's approval before acting. On the loop, the agent acts on its own while a person monitors and can intervene or stop it. In the loop suits actions that are hard to undo; on the loop can suit high-volume, low-risk actions with good logging.

Doesn't asking for approval defeat the point of an AI agent?

No, because the expensive part of most jobs is the work before the decision: reading, checking, comparing, drafting. The agent does all of that; your approval takes seconds. And you don't approve everything forever: reading and drafting run freely, and proven, reversible actions can graduate to standing permissions.

Which AI agent actions should always need approval?

Anything that moves money, deletes data, changes who has access, or can't be undone and is seen by people outside the business: payments, refunds, account changes, mass emails, public posts and production changes.

Can I let an AI agent act without approval?

Yes, for actions that are reversible, low-stakes and proven over time, such as labelling emails or filing receipts, as long as every action is logged and the job has a budget. Widen autonomy one kind of action at a time, based on the agent's record, not on how confident it sounds.