Human review is not free.
Every time an AI agent stops to ask for approval, it spends a little bit of human attention. That sounds obvious until you look at how most agent workflows are designed. The usual pattern is blunt: let the agent work until something looks risky, then ask a person.
That is not a workflow. That is a hope that the reviewer is always available, always alert, and always good at judging risk in the moment.
The research points in a different direction. Human oversight needs a budget.
The approval button is not the hard part
In Oversight Has a Capacity, Emre Turan studies agent action gating for systems that can run shell commands, edit files, or deploy code. The paper's important move is treating oversight as a resource allocation problem, not just a classifier.
Human attention is finite. If the agent escalates too often, the reviewer starts rubber-stamping. In the paper's model, more escalation can make the system less safe because the reviewer becomes part of the failure mode.
That should change how teams write approval gates.
An approval gate is only one control. The harder question is what deserves to interrupt a human now.
Operating Imperfect AI makes a similar point from a queueing angle. Wang and Rachev model human-in-the-loop AI as a system with scarce review capacity, queue backlog, and reliability drift. Static escalation rules miss that reality.
If the agent gets less reliable while the review queue gets longer, the same threshold can be too lax for risk and too expensive for throughput.
This maps cleanly to knowledge-work agents.
A coding agent should interrupt before a production write, secret access, package install, or destructive file change. It should not interrupt every time a linter complains.
A customer operations agent should interrupt before sending a customer-visible message that changes terms, refunds money, or touches regulated data. It should not ask a human to approve every low-risk classification.
A research agent should interrupt when source quality is weak or the claim is unsupported. It should not stop every time it finds two sources that disagree. That belongs in a review queue unless the claim is about to leave the building.
The interruption budget
Keep reading with free field-guide resources.
VibeSec Advisory publishes practical research, Skills, workflow examples, MCP notes, prompt injection tests, and AI red-team lessons for builders working with agentic AI.
The practical artifact is an interruption budget.
It has four lanes.
Interrupt now
Use this for irreversible actions, sensitive data, customer-visible sends, access changes, money movement, compliance-sensitive steps, or suspected prompt injection. The agent stops and waits.
Batch for review
Use this for low-risk uncertainty, repeated minor tool errors, weak classifications, or decisions that need trend review. The agent records the issue and continues only inside the approved boundary.
Stay in shadow mode
Use this when the team is still calibrating the workflow. The agent proposes the action, logs the evidence, and simulates the outcome without write access.
Stop
Use this when the objective is ambiguous, the retry budget is exhausted, the same failure repeats, evidence is missing, the dependency is unhealthy, or the reviewer queue is already over the agreed limit.
That last one matters. A crowded review queue is not just an operations problem. It is a safety signal.
Humans can become part of the failure mode
The older automation-bias literature explains why.
Parasuraman and Manzey reviewed complacency and bias in human use of automation. Wickens and coauthors studied imperfect automation and how people respond after automation failures. The details come from older human factors contexts, but the lesson travels: people can over-trust automated systems, especially after a run of routine correct outputs.
Agent approval queues can train the same behavior.
If a reviewer sees 80 boring approvals in a row, the 81st request does not get a fresh human. It gets a tired human with a learned habit.
That is why "human in the loop" is too vague.
The loop needs a policy:
- What interrupts immediately?
- What waits for batch review?
- What stays in shadow mode?
- What stops the workflow?
- How many approvals can one reviewer handle before quality drops?
- What signal says the queue is overloaded?
- Which approvals get sampled later?
- Which rejected actions update the Skill or guardrail?
Where this fits in the field guide
For VibeSec's field-guide framing, this is a Schedule and Guardrails problem in one artifact.
The Schedule says when the agent may ask, when review happens, and when the loop ends. The Guardrails say which actions require approval, which actions are blocked, and what evidence the reviewer needs.
OWASP's Agentic AI Security Initiative and its agentic AI threats and mitigations guide reinforce the need to map agent actions, risk controls, and oversight before autonomous workflows touch real systems. The NIST AI Risk Management Framework gives the broader governance frame: map, measure, manage, and govern the risk.
The practical next step is smaller.
Do not start with "add a human approval step."
Start with the interruption budget.