Who stops an AI agent when it goes wrong
Main BTU researchers assess that stopping an AI agent safely requires a named person who connects error detection to
Main
BTU researchers assess that stopping an AI agent safely requires a named person who connects error detection to effective intervention and service recovery. Stopping limits further action, but it does not automatically undo an email already sent, an order already changed or information already disclosed.
Key Takeaways
- Stopping an AI agent is an organizational responsibility: the recipient of an alert, the person able to intervene and the person approving a safe return must be identifiable in advance.
- Interrupting the current task, stopping connected work and revoking access have different effects; success at one step does not establish that the entire process is under control.
- For Georgian businesses, a sound basis for using AI agents is the ability to continue a customer’s case after an error, rather than judging the service only by its speed when everything works.
An AI agent can let a business complete a task across several applications without an employee continually transferring the work between them. Such software does more than produce text: using permitted tools, it can retrieve information, change a record or start work in another system. Those connections increase organizational responsibility alongside potential value. When an error occurs at one stage, the important question is who will notice before its effects travel further.
A company may have a capable model, clear instructions and an employee who occasionally checks responses. Yet a practical question can remain unanswered: what does that employee do on finding an error in a process already running? If the answer is simply to email the supplier, the system may continue working until someone intervenes. Having oversight and having the capacity to intervene promptly are different conditions.
Drawing on analysis of BTU research materials and comparison with primary sources, the central finding is that the ability to stop must cover the entire workflow. The responsible person needs to establish what happened, what must not continue and which work a human should take over. This is a governance recommendation, not a claim that systems operating in Georgia already possess these capabilities. [1]
Detection is the beginning of intervention
An agent’s error does not always look like a technical outage. Software can respond quickly in the expected format while writing to the wrong customer or preparing an offer under outdated terms. Watching whether a program remains available will not reveal that difference. The organization needs visibility into the action and its connection to the particular business case.
Employee knowledge matters here. A customer service specialist may notice that a response no longer matches the company’s actual terms. A finance employee may recognize a request that repeats an operation already completed. Those employees need a clear way to suspend questionable work or immediately reach someone who can. Their knowledge has limited protective value if it cannot reach the point where action occurs.
An alert is useful when it has an identifiable recipient and consequence. If it goes only to a shared channel, everyone may assume someone else acted. The responsible person need not build the model or write the software. They need to understand the disruption, initiate the appropriate intervention and know who substitutes for them during an absence. Otherwise, responsibility depends not just on a person’s name but also on their working hours.
Who decides to stop
The US National Institute of Standards and Technology, NIST, connects deactivation mechanisms with clearly assigned responsibilities in the playbook accompanying its voluntary AI RMF 1.0, released in 2023. MANAGE 2.4 addresses superseding, disengaging or deactivating systems whose performance or outcomes diverge from their intended use. This is governance guidance, not a guarantee that a product is safe. [2]
The Government of Canada’s guide on agentic AI adds a concrete point: a pause and disable mechanism should exist outside the agent or agentic system, alongside a recovery plan. The guidance is intended for Canadian federal organizations, not a rule applicable in Georgia. The useful business principle is that stopping a problematic agent should not depend solely on that agent responding correctly. [6]
For a business, these principles make the decision clearer. A technical specialist may be able to disconnect access without understanding the consequences for a customer’s unfinished case. A process manager may understand those consequences but lack the ability to intervene in the system. An incident therefore needs a named lead who connects business knowledge with the capacity to act. Technical work can be distributed across several people without leaving leadership of the incident unclear.
Temporarily pausing a questionable process and permanently retiring an agent are different decisions. A rapid restriction in response to a credible threat should not wait for a complete investigation. At the same time, an automatic pause can unnecessarily interrupt service. Preapproved conditions for an automatic response and subsequent human judgment therefore complement each other. The responsible person determines when a temporary restriction is sufficient and when broader intervention is needed.
Supplier involvement does not remove this responsibility. A company needs to know which actions it can stop itself and which depend on external support. When a customer receives a service from a Georgian business, that business should remain an intelligible point of contact during an incident. Waiting for a supplier’s reply cannot substitute for explaining the status of the customer’s case.
What actually stops
An agent appearing to finish on screen may mean only that the current conversation has ended. Work already passed to another application may remain a separate process. At the moment of interruption, for example, a message may still be in a sending queue while an external system has already accepted an order change. These are possible architectural differences; their presence in any particular product requires verification.
The effect of stopping must therefore be checked in the systems where the agent acted. An accepted request and a confirmed completed action are different states. A label saying “cancelled” may not answer what happened to work started earlier. Without that information, an employee can restart the case and inadvertently repeat an operation already completed.
Revoking access is another distinct issue. In many systems, software connects to a service using a digital credential called a token. The OAuth revocation standard recognizes that information about invalidation can propagate between servers with a delay; whether related tokens are also invalidated depends on policy and implementation. Sending a revocation request therefore cannot establish that every connection has stopped immediately. [3]
Different effects of intervention
| Intervention | Intended purpose | Does not establish by itself |
|---|---|---|
| Interrupt the current task | Stop the agent’s next steps | That work started elsewhere has stopped |
| Cancel connected work | Stop a particular queue or external task | That completed actions have been undone |
| Revoke access | Restrict subsequent use of a service | Immediate disconnection of every connection |
| Restore service | Continue the case in an agreed safe state | An exact return of every consequence to the original state |
Source: BTU qualitative analytical synthesis [1–6]. The comparison describes distinct purposes, not guaranteed capabilities of a tested product.
These distinctions explain why stopping needs an accountable person. Disabling a program can succeed technically while the business still lacks an answer about connected tasks that remain active. A complete picture is particularly important when parts of the work have been handed to different employees, suppliers or agents.
Stopping does not erase what happened
Microsoft’s architecture guidance treats correction of completed steps in distributed processes as a separate problem. Such correction may not restore the original state: other changes may have occurred, and cancellation can have its own business conditions. Corrective operations can also fail. This is a general software architecture issue that applies to agents acting through tools as well. [4]
Consider an illustrative Georgian travel company. An agent starts changing a booking and sends the customer new terms. An employee spots a date error and stops the agent. Further actions need to cease, but someone must now establish whether the hotel accepted the change, what the customer was told and whether the original booking remains valid. The example explains the mechanism; it is not a report of an actual incident.
Replacing a record with an older copy may not solve the problem. The customer may already have changed plans, while the partner may have allocated capacity to another request. Correction then means reconciling the actual situation with the parties, explaining the error and choosing an acceptable resolution. The economic cost extends beyond program running time to customer time, additional employee work and confidence in the service.
Preserving a trace of actions is therefore essential. What the agent requested, what the external system accepted and what was finally confirmed may be different records. An investigation needs to connect them without collecting or retaining unlimited personal information. Access should be restricted to people who need the information to investigate and correct the incident.
Bringing service back safely
After an error, the organization has two parallel tasks: continuing the customer’s case and correcting the problematic system. The first need not wait for the second to be fully resolved if manual service can proceed safely. A human handover requires a clear state: what is complete, what remains uncertain and which action must not be repeated.
NIST’s 2025 incident response recommendations connect detection, response and recovery. Recovery includes confirming that systems function normally. This is a cybersecurity framework; applying it to an agent’s substantive mistake also requires checking the business outcome. A technically healthy connection cannot establish whether a customer’s changed order is correct. [5]
Returning to an older version is only part of the answer. If the error arose from shared data or a change in an external system, the older software may face the same problem. If incorrect information remains in the agent’s working memory, a fresh run may use it again. A safe return therefore requires a correction matched to the cause and a check of the case that exposed the failure.
The person approving restart needs to see the basis for that correction. “It works now” does not establish that the cause has been addressed. A restricted return with observation of results can reduce uncertainty without placing the whole service on an untested change at once. This is a recommendation for verification, not a promise of zero risk.
What matters for Georgian businesses
The issue becomes particularly concrete when a Georgian company communicates with customers in one language while supplier support operates in another. Accurately conveying the problem, interpreting dates and terms consistently, and confirming completed actions become part of incident management. Those dependencies should inform service selection, although this analysis does not measure their effects.
In a small organization, one person may hold several roles. A real ability to stop and an available substitute matter more than creating a large department. A clear route for continuing service lets the company pursue AI’s benefits without facing only two choices when an error is found: complete inactivity or an indefinite wait.
A person designated by the organization is therefore accountable for stopping the agent. A preauthorized automatic mechanism or a technical team may perform the actual interruption. That person’s responsibility does not end when the interruption is initiated. The organization needs confirmation that unwanted action is no longer continuing, the effects already produced are understood and the customer’s case can move safely to its next stage.
Questions and answers
Can an agent stop itself?
Automatic stopping under predefined conditions is possible, but responsibility for assessing the incident and approving a safe return should remain with the organization.
Does revoking access undo actions already completed?
No. Revocation limits subsequent use of access; correcting completed actions requires separate assessment and action.
Must every error shut down the entire system?
No. Intervention should reflect potential impact and spread; isolating a particular task may sometimes be enough.
When can the agent restart?
When the responsible person has a verified basis for the correction, the state of ongoing cases is known and the results of the return can be observed.


