Human-in-the-Loop AI Agents: Approval Gates That Work

Every team building an agent eventually has the same conversation. The agent can draft the email, but should it send it? It can write the migration, but should it run it? It can find the duplicate customer records, but should it delete them?

The usual answer is "a human should approve it". That is right, but it leaves the hard questions open. Which actions need approval? How do you pause an agent mid-task and resume it hours later? What should the approver actually see? This article covers the design decisions, with code for the Claude API.

The two ways to get it wrong

Approval gates fail in two opposite directions.

Too few gates and the agent takes an action nobody intended: a message to a customer based on a misread instruction, a deleted file, a published draft. The model may be right 99 percent of the time. The 1 percent is what people remember.

Too many gates and approvals become noise. If a person has to click "approve" thirty times an hour, they stop reading what they approve. A gate that is always approved without reading is worse than no gate, because it gives a false sense of safety. This is approval fatigue, and it is the more common failure in practice.

The goal is a small number of gates, each on an action where a human decision genuinely adds something.

Classify tools by risk, not by feature

The cleanest way to decide is to sort every tool the agent can call into three tiers:

• Read. Searching, fetching, listing, reading files. No side effects. Never gated.
• Reversible write. Creating a draft, writing to a scratch branch, adding a label, saving something that can be undone in one step. Usually not gated, but logged.
• Irreversible or outward-facing. Sending, publishing, paying, deleting, changing access, anything other people will see or that cannot be cleanly undone. Gated.

Two questions decide the tier: can this be undone? and will someone outside the system notice? An action that fails either test belongs in the third tier.

The classification belongs in code, next to the tool definitions, not in the system prompt. A prompt that says "always ask before sending email" is a request to the model. A gate in the loop is a guarantee.

type Risk = 'read' | 'write' | 'gated';

const TOOL_RISK: Record<string, Risk> = {
  search_tickets: 'read',
  get_customer: 'read',
  draft_reply: 'write',
  add_label: 'write',
  send_reply: 'gated',
  refund_order: 'gated',
  close_account: 'gated',
};

function needsApproval(toolName: string): boolean {
  // Unknown tools are treated as gated: fail closed.
  return (TOOL_RISK[toolName] ?? 'gated') === 'gated';
}

Note the default. A tool that someone added without classifying it is treated as gated. That way a forgotten entry costs an extra approval rather than an unreviewed action.

Pausing the loop

An agent loop normally runs like this: send the conversation to the model, execute any tool calls it makes, append the results, repeat until the model stops calling tools. (Our agent loop explainer covers the basics.)

To add a gate, check each tool call before executing it. If it needs approval, stop the loop and save everything needed to resume: the full message history and the pending tool call.

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic();
const MODEL = process.env.AGENT_MODEL; // whichever Claude model you deploy

async function runStep(run: AgentRun) {
  const response = await client.messages.create({
    model: MODEL,
    max_tokens: 4096,
    system: run.systemPrompt,
    tools: run.tools,
    messages: run.messages,
  });

  run.messages.push({ role: 'assistant', content: response.content });

  const toolCalls = response.content
    .filter((b) => b.type === 'tool_use');
  if (toolCalls.length === 0) return { status: 'done' };

  const gated = toolCalls.filter((c) => needsApproval(c.name));
  if (gated.length > 0) {
    run.pending = toolCalls;          // keep the whole batch together
    run.status = 'awaiting_approval';
    await saveRun(run);               // durable storage, not memory
    await notifyApprover(run, gated);
    return { status: 'paused' };
  }

  const results = await Promise.all(toolCalls.map(executeTool));
  run.messages.push({ role: 'user', content: results });
  await saveRun(run);
  return { status: 'continue' };
}

Two details matter here. First, the run is saved to durable storage, because an approval might take minutes or days and the process will not still be running. Second, if the model made several tool calls in one turn, all of them are held together. The API expects a result for every tool call in the previous turn, so you resume with the whole batch.

Resuming after a decision

When the approver decides, you load the run, execute or refuse each pending call, and send the results back. The model then continues with full knowledge of what happened.

async function resume(runId: string, decision: Decision) {
  const run = await loadRun(runId);
  const results = [];

  for (const call of run.pending) {
    if (!needsApproval(call.name) || decision.approved) {
      const input = decision.editedInput?.[call.id] ?? call.input;
      results.push(await executeTool({ ...call, input }));
    } else {
      results.push({
        type: 'tool_result',
        tool_use_id: call.id,
        is_error: true,
        content: 'Rejected by reviewer: ' + decision.reason,
      });
    }
  }

  run.messages.push({ role: 'user', content: results });
  run.pending = null;
  run.status = 'running';
  await saveRun(run);
}

A rejection is returned as a tool result marked is_error with the reviewer's reason. That is important. If you simply drop the call, the model has no idea why its plan stopped working. With the reason, it can adjust: rewrite the reply in a different tone, ask a clarifying question, or stop and report.

What the approver needs to see

The quality of an approval depends almost entirely on what is in front of the reviewer. A notification that says "Agent wants to call send_reply. Approve?" is useless. They need:

• The exact action. The full text of the message, the exact amount of the refund, the list of records to be deleted. Not a summary written by the model.
• The target. Who receives it, which account, which environment.
• The reason. The model's short explanation of why it is taking this step, and the original request it is working on.
• Edit as well as approve. Often the right answer is "yes, but change this line". Letting the reviewer edit the tool input before approving avoids a frustrating round of rejections.

Render the raw tool input, not the model's description of it. The point of the gate is to check what will actually happen, and a description can differ from the action.

Timeouts default to no

Approvals do not always come. The reviewer is on holiday, the notification was missed, the request was ambiguous. Decide in advance what happens then, and make the answer "the action does not happen".

Set an expiry on every pending approval. When it passes, resume the run with a rejection whose reason is "approval timed out". The model can then tell the user the task is incomplete. An agent that proceeds because nobody objected is not human-in-the-loop. It is a delay timer.

Reducing approval volume safely

Once the basic gate works, you will want fewer approvals. A few techniques help without weakening it:

• Batch related actions. Ten replies to similar tickets can be reviewed on one screen and approved together.
• Scope by value. A refund under a small threshold might be a reversible write, while one above it is gated. Put the threshold in code.
• Draft first, act second. Many gated tools can be split into a write-tier "draft" and a gated "send". The agent prepares everything; the human approves the final step only.
• Earn autonomy with data. Log every approval decision. If a tool has been approved unchanged hundreds of times, that is evidence for moving it down a tier. If reviewers often edit it, it should stay gated.

Keep an audit trail

Record who approved what, when, and with what edits, alongside the agent's reasoning and the final tool result. When something goes wrong, and eventually something will, this is how you find out whether the model made a bad proposal or a human approved one without reading it. Our article on agent tracing and observability covers how to store this alongside the rest of a run.

Gates are one layer

Approval gates protect against an agent doing the wrong thing with the tools it has. They do not protect against an agent having tools it should never have had. Combine them with least-privilege permissions, so the worst case is limited even before a human looks, and with idempotent tool calls, so that an approved action is not accidentally executed twice when a run resumes after a crash.