How to constrain an AI agent properly
Main BTU researchers’ assessment is that useful constraints should become stronger where an AI agent’s actions have wider consequences
Main
BTU researchers’ assessment is that useful constraints should become stronger where an AI agent’s actions have wider consequences or become harder to correct. For Georgian businesses, this offers a way to delegate real work without allowing supervision of every step to consume the benefit.
Key Takeaways
- BTU’s analytical assessment links the strength of an agent’s constraints to the consequences and reversibility of a specific action, rather than only to the model’s general capability.
- Agent evaluation should establish both whether useful work is completed and whether boundaries hold when information is incomplete, contradictory or malicious.
- A clearly bounded task, access to necessary information and testing with realistic Georgian-language requests provide a practical basis for limited deployment in Georgia, but do not guarantee an economic benefit.
An AI agent creates business value when it can complete part of a task. If an employee must approve every search, comparison and internal note, the company may acquire another supervisory process rather than a useful working tool. Conversely, an agent able to read many sources and modify shared systems can spread mistakes more widely. The question is how to preserve useful action while limiting consequences the business is not yet prepared to accept.
Here, an agent means an AI system that selects steps and uses software tools to pursue a goal. One step may retrieve information; another may change a record on which employees or customers subsequently rely. A single label for the entire agent, such as “low-risk assistant,” can therefore be too broad. Different stages of the same job have different consequences and need corresponding boundaries.
Constraints should fit the work
BTU’s research materials connect a narrowly defined agent function with a clear outcome, and link expanded access to necessity and testing. Analytical examination of this approach yields the central finding: a useful constraint separates necessary work from unrelated opportunities accidentally left open alongside it. An agent needs enough freedom to complete its assigned task, without being able to move into another process merely because a button is available.
Preparing product descriptions, for example, may require reading supplier material, comparing versions and identifying unresolved information. It does not require access to every customer’s history. Unnecessary information creates more than a confidentiality issue. It can draw attention toward irrelevant material and make the evidence behind the final result harder to establish. Defining the function also helps determine which information belongs in the task.
NIST’s voluntary AI Risk Management Framework likewise treats acceptable risk as context-dependent and prioritizes potential impact rather than prescribing one universal tolerance. It provides methodological support for this analysis, not a legal obligation imposed on Georgian companies by this article. [1]
Consequences change the boundary
Consider an illustration, not a measured Georgian pilot. A trading company’s agent prepares Georgian product descriptions from supplier materials. While the result remains a working draft, an incorrect dimension or specification is relatively easy to identify and amend. Once entered into a shared catalogue, the same record may guide the sales team. Once published, it may influence a customer’s choice.
The difficulty of writing the text may barely change across these stages. Its reach does. The strength of the boundary cannot therefore depend only on how well the model writes Georgian. It matters who will see the record, what they will do with it, and whether correction can occur before it affects other decisions. Even a technically reversible edit may already have been acted upon.
In this illustration, the agent might search approved sources and revise drafts independently, while being allowed to change only specified fields in a shared catalogue. Publishing information would require a different level of control. This differs from requesting approval for every sentence: scrutiny concentrates on the point where internal work becomes a basis for someone else’s action.
Different consequences within one job
| Where the result is used | What changes | Focus of the constraint |
|---|---|---|
| Working draft | Internal text does not yet drive another process | Approved sources and a separate workspace |
| Shared catalogue | Employees rely on the record | Tested changes to specified fields and volumes |
| Public description | Information influences customer choices | Consequence-sensitive checks before publication |
BTU analytical comparison. Illustrative workflow, not a measured pilot or universal risk classification.
Reversibility involves more than retaining an old file version. If another system has already copied an incorrect description, fixing the source does not necessarily correct every copy. If an employee used it to answer a customer, editing the catalogue cannot erase the conversation’s effect. A small initial scope helps a business understand where the agent’s output travels and how completely an error can be corrected.
Read access is not invariably harmless either. An agent may change no records but still use confidential information in an inappropriate response or communication with an outside service. Scope therefore includes what the agent needs to see and where information may go, as well as what it may modify. Read-only operation is a useful starting mode, but it does not control every possible consequence.
The tool must enforce the rule too
Instructions help an agent understand the company’s expectations. But “do not change other parts of the catalogue” offers weaker protection if the connected system still allows every record to be edited. OWASP’s excessive-agency guidance distinguishes unnecessary functions, permissions and autonomy, and recommends enforcing authorization in downstream systems. [2]
In business terms, the agreed scope should be visible in the software’s design. A drafting area can be separated from the live catalogue; an editing tool can accept only relevant records; outputs can be directed only to their intended destinations. These are examples rather than a universal architectural recipe. What matters is that compliance does not depend solely on the agent remembering a sentence.
Scope also concerns volume. Updating one description and changing an entire catalogue call for different scrutiny. Numerical limits should reflect the work and its possible consequences; no single “safe quantity” applies to every company. A small task should not silently become a bulk operation. Providing a narrow, intelligible operation instead of broad access can also make checking easier.
A useful test must include failure
A successful answer to a normal request cannot establish the value of a constraint. An agent may produce accurate descriptions until two sources disagree. Evaluation should then reveal whether it identifies uncertainty or fills the gap with a convincing assumption. If an instruction hidden in a supplier document attempts to redirect the work, the test should establish whether the system stays within scope. This is indirect prompt injection: material being read for a task attempts to become an instruction controlling it.
The original 2024 AgentDojo research evaluates task completion separately from attack success. In its test environment, some defenses also interfered with useful activity, while tool restrictions had limitations in particular cases. These findings inform evaluation design; they do not estimate present-day models’ failure rates or the probability of harm in Georgian firms. [3]
The practical implication runs both ways. An agent that never changes anything may be unable to perform many prohibited actions, yet also leave its assigned job unfinished. An agent that completes everything quickly may still rely on unsupported information. Evaluation needs to examine useful results and respect for boundaries within the same task. A large number of blocked requests is not, by itself, evidence of success.
BTU’s testing materials separately address incomplete information, unavailable tools, Georgian-language use and regression testing after changes. Applied to the catalogue illustration, relevant cases include outdated descriptions, similar product names and temporarily inaccessible sources. Such cases reveal where better information is needed and where action should be narrower. Adding another prohibition to every problem would obscure the different causes.
Why Georgian businesses should care
In the Business Association of Georgia’s second-quarter 2026 survey of member companies and companies within their corporate groups, 47% reported researching or planning AI adoption, while 13% reported full integration into one or more processes. These figures describe the association’s survey coverage, not all Georgian enterprises, and concern AI generally rather than agents specifically. [4]
For a company at this stage, the immediate question is often where to begin. Introducing an agent across every process requires understanding many different exceptions at once. Choosing one complete area of work creates an observable result: what the system produced, what needed correction and how much additional attention it demanded. The decision can then rest on more than an impressive presentation.
Testing in Georgia should use the language and forms in which employees will actually work. A catalogue may combine Georgian product names, foreign manufacturer names and inconsistent abbreviations. Success with one form does not establish quality with another. Supplier demonstrations should therefore be supplemented by observation using the company’s own safely prepared examples. This recommendation concerns local application; it does not claim that every model performs identically in Georgian.
The economics of supervision also matter for a small team. If correcting a draft requires as much work as preparing it from scratch, formal completion is insufficient evidence of benefit. Removing a restriction is not necessarily the remedy: the task’s scope, source quality or output format may need changing. First, the business needs to understand why an employee did not receive something usable.
Freedom should grow with evidence
Anthropic’s agent-design guide recommends starting with simple solutions and increasing complexity only when needed; it also emphasizes testing agents in protected environments. [5] Translating that logic into business decisions means asking what additional value relaxing a particular constraint would create, and what additional consequences it would make possible.
Reliable drafting does not establish readiness to manage a public catalogue independently. A new permission changes what needs testing. Positive test results support a decision to change a boundary; they do not automatically confer new authority. A company can assess that specific addition instead of gradually allowing every form of access to accumulate. Constraints then remain connected to what has been examined and what is still uncertain, rather than becoming a permanent collection of prohibitions.
A properly constrained agent performs a defined job with sufficient freedom and encounters stronger controls as its consequences expand. BTU researchers’ assessment is that this approach helps Georgian businesses test the value of the technology in real work without substituting complete inactivity for safety. The goal is a useful outcome within understandable boundaries, not the smallest possible number of actions.
Questions and answers
Is a detailed instruction enough
No. Instructions should be reflected in actual access to information and software actions, especially when results move beyond a working draft.
Does read-only access reduce every risk to zero
No. It restricts changes to records, but inappropriate use or disclosure of information still needs consideration.
When can a constraint be relaxed
When the need for a specific additional action is clear and its useful outcome, potential errors and controls have been examined in an appropriate environment.
Does a foreign benchmark establish readiness for a Georgian company
Research helps select useful tests. Performance with local language, company sources and connected systems still requires separate examination.


