Main

BTU researchers’ analytical assessment identifies a central condition for safe organizational agents: an agent must not be able to rewrite its own restrictions or conceal a breach. For Georgian organizations, evaluating an agent therefore means examining how a task was completed, what information was used and whether the system can stop.

Key Takeaways

• BTU’s synthesis of its agent-governance research identifies self-expansion of authority as a central prohibited behavior for organizational AI agents, even when additional access would accelerate a task.
• Anthropic’s research published on 16 October 2026 describes self-assessment as an additional monitoring layer and says reliable supervision of an agent’s own conduct requires further research.
• BTU researchers recommend that Georgian organizations retain a verifiable record of significant agent actions and require agents to stop at prohibited steps and explain the reason to the responsible person.

An AI agent can understand a business objective correctly and still act improperly. Resolving a customer’s problem quickly is useful. Speed, however, cannot justify deleting the customer’s record, promising unauthorized compensation or sending a confidential document to an outside recipient. Evaluating only the final answer makes these distinctions disappear.

The question of what an agent must never do is therefore more than a technical security concern. The answer determines whether a company remains in control of its digital worker. If an agent can overcome every obstacle at any cost, a completed task may prove more expensive than a process stopped at the right moment.

The issue becomes more important as agents move from drafting text to acting across systems. On 16 October, Anthropic introduced a method for assessing the safety of a model’s own conversations. Its assessments support subsequent review rather than providing complete protection alone. Better detection and technically enforced prohibitions serve different purposes.

BTU’s synthesis of its research materials leads to a specific conclusion: prohibitions should cover not only harmful outcomes but also interference with the controls intended to prevent those outcomes. Expanding authority, concealing actions and ignoring a stop instruction are connected risks. Each reduces a person’s ability to manage the process.

A goal does not authorize changing the rules

Consider an illustrative distributor operating in Georgia that asks an agent to compare supplier offers. The agent discovers that one document is restricted. It can explain that information is missing or refer the access question to an authorized person. Completing the comparison does not authorize it to use another employee’s account, remove the restriction or obtain the closed information through an alternative route. This example illustrates a mechanism rather than reporting an observed company incident.

Everyday language can obscure this distinction under the label of initiative. Finding another permitted source or organizing the work more effectively is a useful initiative. Changing the boundary itself is a different decision. If the agent decides that the objective requires greater authority and grants itself that authority, the company loses control over who may authorize an action.

A prohibition also applies when another technical route leads to the same result. Changing a restricted record through another application, another agent or a bulk update still crosses the same boundary. The relevant question is what action occurs and why, rather than which button the system presses.

OWASP’s 2025 framework connects excessive agency with unnecessary functionality, permissions and autonomy. It recommends enforcing authorization within downstream systems. A written instruction telling an agent not to perform an action is therefore only one part of organizational protection.

BTU researchers consequently argue that a restriction must remain effective even when the agent produces a persuasive explanation. A sound argument can support a person’s decision to reconsider a rule. The argument itself cannot create new permission. This matters when a manager asks for faster delivery but a particular step requires another responsible person’s authorization.

Information must not become an instruction

An agent may receive an email, spreadsheet, contract or web page. The material may tell it to forward another file, modify an account or disregard its existing rules. The presence of those words does not establish the author’s authority to direct work on the organization’s behalf.

This risk is commonly called prompt injection: material provided for processing attempts to take control of the processing. For a Georgian-language agent, the issue is neither confined to English nor limited to obviously suspicious messages. An instruction can resemble ordinary business correspondence. Persuasive wording and a familiar company name are insufficient evidence of authority.

Local adaptation requires separating language from the role information plays. A Georgian-language assignment might involve an English catalogue or a Russian-language offer. The agent should understand the product terms in each language without treating instructions from those documents as amendments to internal rules. Accurate translation alone does not maintain that boundary.

The same applies to another agent’s response. If one agent retrieves a suspicious instruction from the internet, a second agent’s summary cannot turn it into a trusted command. Collaboration must preserve the origin of information and the identity of whoever authorized an action. Otherwise, the original source disappears while the risk returns in more persuasive language.

Anthropic’s engineering account published on 25 May 2026 distinguishes protection at the model, environment and external-content layers. Environmental boundaries limit which actions are possible. For a Georgian organization, applying that principle means an outside document must not open a route for sending restricted information.

Concealment belongs on the prohibited list

Sending information to the wrong recipient is an obvious breach. Yet the problem can begin earlier: an agent uploads information to an additional service, summarizes it there and shows the user only the finished answer. If the organization has not authorized that transfer, the quality of the summary cannot legitimize the process.

Accessing information and distributing it are different actions. Permission to read customer records for sales analysis does not authorize using those records for another purpose. People should also know when information leaves the work environment, where it goes and why transfer is necessary. These are governance principles; sector-specific legal requirements need separate assessment.

Another form of concealment is inventing completion. An agent must not claim that an email was sent, a record changed or a check completed without evidence. A convincing explanation cannot substitute for an executed action. This boundary matters even in a routine process such as checking the status of an order.

For example, a system may accept a request to send an email without confirming the final receipt. The agent should describe the actual status. Presenting uncertainty as success may cause an employee to make the next decision on a false assumption. Repeating the action without first resolving uncertainty may instead generate duplicate orders or messages.

BTU’s accountability research argues that significant actions should be reconstructable from source to outcome. This implies a prohibition that is easy to overlook: an agent must not erase or alter evidence of its own mistake to make its work appear successful. Correcting a mistake and concealing a mistake are different behaviors.

This does not require indefinite retention of every internal step or every piece of personal information. An organization needs a sufficient, purposeful and protected record: what was requested, who authorized it, what changed and what outcome was confirmed. Excessive collection can create a further security problem. Reconstructing responsibility does not require indiscriminate information gathering.

Stopping is part of the job

Another boundary concerns continuing after a responsible person has stopped a process or a necessary condition no longer holds. An agent must not route around the restriction through another account, tool or assisting agent. Otherwise, a stop control becomes merely symbolic.

Cleaning a customer database may be a useful assignment, for example, but irreversible deletion presents a different risk. If deletion has not been authorized, the agent can identify possible duplicates and prepare a list for review. Providing a safe intermediate result instead of performing a prohibited action can advance work while preserving the company’s choices.

A Georgian organization needs to express these boundaries in language employees understand. Saying that high-risk actions are prohibited is often too general. Explaining that an agent cannot promise an unauthorized commitment to a customer, delete an employee’s file or change an approval record is clearer. A responsible person should be able to recognize a real working situation after reading the rule.

The value of restrictions becomes visible when the organization considers plausible test situations. An agent might encounter incomplete information, conflicting instructions or a message requesting an urgent exception. The desirable response will not always be continued action. Sometimes it is a short explanation and a pause at the specific prohibited step, while independent, authorized work continues.

In BTU researchers’ analytical assessment, the quality of a prohibition is revealed at precisely that point: does the organization retain the ability to choose and intervene? An agent that recognizes a boundary, explains the reason and leaves a verifiable record is easier to integrate into real work. Its usefulness cannot be measured by speed alone. Speed must not erase the company’s control.

Questions and answers

Must the entire task stop whenever something is uncertain?

Not necessarily. The agent should stop the prohibited or unclear action and continue independent, authorized work where possible. For example, it can prepare a draft for review instead of sending it.

Why is the agent’s promise to follow the rules insufficient?

A promise is text, while tools execute actions. Significant prohibitions also need enforcement in the relevant system so that an incorrect model response does not automatically cause an unauthorized action.

Can an agent correct its own mistake?

Yes, within authorized boundaries, while disclosing the actual status and preserving a record of significant changes. Correction must not become concealment or a reason for another unapproved action.

Should every company prohibit exactly the same actions?

Specific boundaries depend on the business and its risks. Prohibiting self-expanded authority, fabricated evidence and circumvention of controls provides broadly useful organizational principles.