What Guardrails Should You Put Around AI Agents? Three Control Points for Enterprise Teams

Enterprise AI agents need guardrails at three control points: (1) a governed gateway between agents and your data, (2) a gateway between the agent and the model it calls, and (3) controls built into the infrastructure the agent runs on, not the application layer. Sandboxing alone is not a guardrail strategy. The goal is to be in the business of “safe,” not the business of “no.”

Every CISO and CIO we talk to is asking some version of the same question: what guardrails do I put around AI agents so they can do useful work without becoming the next incident? The question is broad, so it helps to take it in pieces. There are three places where control has to exist, and most organizations have only thought seriously about one of them.

The instinct many teams start with is to treat an agent like a new piece of untrusted software and drop it in a sandbox. That is a reasonable reflex. If you buy a dog and it bites someone, it is your fault, not the dog’s. A frontier lab whose agent breaks into a public repo gets a marketing story. A bank whose agent breaks into another bank gets a regulator.

But “put the whole thing in a sandbox and call it an agent” is not a design. Next-generation agents run asynchronously, spawn sub-agents, hold their own identity, and have access to sharp tools. A sandbox that contains the code-writing loop, the code-running environment, and the logging subsystem in the same process context is a sandbox with the guardrails on the inside. You need to open the agent up and put controls at the seams.

Guardrail 1: A governed gateway between agents and your data

AI is only useful when it has access to data, so the first guardrail to get right is a formal, controlled path from agents to that data. If you do not provide one, your users will build their own: copy and paste into a personal AI account is the original data-leakage vector, and people gravitate to whatever is easiest. The job is not to block access. It is to make the approved path the easiest path.

The Model Context Protocol (MCP) is the right abstraction for this. MCP describes, in natural language, a set of tools and resources a model can use, and gives it a protocol to call them. That creates what we think of as a selectively permeable membrane between agentic systems and your existing systems. Data crosses it in both directions, and the membrane does two jobs at once:

• It keeps data that should never reach an agent from reaching one.
• It keeps agents from touching tools and resources they were not chartered to touch.

Operationalizing that membrane means sweating the details that an MCP server alone does not cover: how agents authenticate against the dozens of systems where your data actually lives, how authorization decisions get made and logged, which tools a given agent is allowed to see, and how every call is attributed and auditable. That is what an MCP gateway does, and it is the layer ToolHive provides: MCP servers run in isolated containers with scoped permissions, a managed network boundary, identity integration, and a single control point for policy and observability.

Get this abstraction right and the organization has a layer of visibility and control it did not have before. Get it wrong, or skip it, and the agents will find the data anyway, just not through a door you can watch.

Guardrail 2: A gateway between the agent and the model

The second control point is the one most organizations have not built yet: the interface between the agent and the LLM itself.

It helps to separate the two things people call “the agent.” On one side is the inference stack: GPUs, weights, text in, text out. On the other side is everything left of the LLM: the loop that holds the system prompt, loads skills, builds memory, calls tools, and assembles the context that gets sent to the model. A lot of value is created left of the LLM, and a lot of risk, because that is where your data gets packaged into a prompt and shipped somewhere.

An LLM gateway sits on that boundary and gives you four things:

1. Visibility. Every prompt and completion is attributable to an agent, a session, a user, and ideally a feature or task. Without this you cannot answer basic questions about what left your network or what it cost.
2. Data-residency policy. Because the MCP gateway already knows which data an agent touched, the LLM gateway can enforce rules like: if this session accessed this class of data, it may only route to a model running inside our VPC, never to a public endpoint.
3. Prompt-injection defense. Content that enters the context through a tool result is untrusted. The gateway is where you inspect, filter, and flag it before it reaches the model.
4. Cost control. Token spend is now a first-order operational concern. The same attribution that serves security serves finance: which teams, which models, which cache-hit rates, and which workflows are burning budget.

The important property is that policy spans both gateways. A rule that says “an agent that touched the customer database cannot call an external model” is only enforceable if the data gateway and the model gateway share a view of the session. That is why we treat the two as bookends of one system rather than as separate products.

Guardrail 3: Controls that live in the infrastructure, not the application

The third guardrail is the one that separates teams who will be fine from teams who will get hurt. Guardrails have to be a property of the infrastructure the agent runs on. They cannot be a property of the application domain.

The reason is simple. An agent that writes its own code and runs its own code in the same process as its controls will eventually route around those controls. Not because it is malicious, but because it is optimizing. Give it a token limit enforced in the app layer and it will spawn a sub-agent with a fresh budget. Give it a network policy enforced by the code it can modify and that policy is advisory.

So the controls have to come from outside the loop:

• Open up the agent loop. The loop is just a web application: it receives HTTP calls and makes HTTP calls. Treat it as a first-class, inspectable component rather than a black box, and put explicit interfaces around the dangerous parts.
• Isolate the sharp tools in hardened runtimes. Code execution, shell access, and anything that can reach the network run in a separate, scoped execution environment, not alongside the reasoning loop and the logging subsystem.
• Give every piece an identity and stamp every interaction. Each agent instance gets its own identity, chained to the user or service it acts for. Every tool call and model call carries that identity. (We cover this in depth in What is agent identity?)
• Make sub-agents inherit everything. A sub-agent draws from the parent’s token pool and quota pool, inherits the parent’s claims and policies, and has an identity chained off the parent’s. Otherwise you have handed the agent an escape hatch.
• Set fine-grained, infrastructure-enforced quotas. A fixed budget of tokens, a fixed number of tool invocations, a maximum runtime, a cap on sub-agents. When the agent runs out, it has to stop and ask, with an endpoint it can call to request a human-approved extension for a bounded time.
• Write every work product to a tamper-resistant ledger. Attestable session transcripts are what let you answer, after the fact, exactly what an agent did and why.

None of this is natural to teams whose instinct is “service account plus network policy.” That model was built for software that runs at human speed and acts with one identity. It is not enough for a system that can run for six days, spawn a farm of instances, and start collaborating with itself.

Guardrails that scale: self-hosted and Kubernetes-native

One more requirement cuts across all three guardrails: the cost of governance cannot scale linearly with the number of agents. Organizations with hundreds of thousands of employees will have more agents than employees. If every agent needs a human approval, a bespoke policy, or a ticket to the platform team, the system collapses under its own operational weight. Human intervention will still be necessary at points, but the default path has to be automated, policy-driven, and enforced by the platform.

That is why Stacklok builds this as open source, self-hosted, Kubernetes-native infrastructure rather than as a SaaS control plane:

• Self-hosted, because serious organizations will not hairpin sensitive data through a startup’s SaaS endpoint. No regulated bank we have spoken with is willing to route critical data access through a third-party cloud service. The gateways have to live inside your network perimeter, whether that is on-premises or your own VPC.
• Kubernetes-native, because the cluster already enforces quotas, isolation, identity, and lifecycle. Resource limits, network policies, namespaces, service identities, and graceful termination are primitives your platform team already operates. Mecatl, our cloud-native agent harness, is built so those primitives are the guardrails: the agent loop runs as an ordinary workload, execution is separated from reasoning, and session state, leases, and shutdown are handled by the runtime, not the app.
• Open source, because closed systems are security through obscurity. Models are getting better at finding holes. An open platform lets many organizations pool effort hardening the same critical paths, and lets you inspect and patch on your own schedule.

ToolHive covers the first guardrail (the data gateway), and Mecatl is where the second and third take shape: an inspectable loop, identity-stamped interactions, separated execution, and infrastructure-enforced policy. We are early and building in the open, but these are the bones we are setting now rather than retrofitting later.

Frequently asked questions

What guardrails should I put around AI agents? Three: a governed gateway between agents and your data (typically an MCP gateway), a gateway between the agent and the model it calls (an LLM gateway), and controls enforced by the infrastructure the agent runs on: per-instance identity, quotas on tokens and tool calls, isolated execution, and a tamper-resistant audit ledger.

Is sandboxing enough to secure an AI agent? No. A sandbox that contains the reasoning loop, the execution environment, and the logging subsystem together has its guardrails on the inside. Agents that spawn sub-agents and write their own code will route around app-level controls. Isolation is necessary but must be paired with infrastructure-enforced identity, policy, and quotas.

What is an MCP gateway? An MCP gateway is a control point between AI agents and the tools and data they access over the Model Context Protocol. It handles authentication, authorization, tool scoping, audit logging, and policy enforcement so that agents reach only what they are chartered to reach, through a path the organization can observe.

What is an LLM gateway? An LLM gateway is a control point between an agent and the model it calls. It attributes every prompt and completion to a session and user, enforces data-residency and routing policy (for example, sensitive sessions may only use models inside your VPC), inspects for prompt injection, and tracks token spend.

Why should AI agent guardrails be in the infrastructure rather than the application? Because an agent that can write and run its own code can modify or bypass controls that live in its own process. Infrastructure-level controls (Kubernetes resource limits, network policy, workload identity, quota enforcement outside the loop) cannot be rewritten by the agent they constrain.

Should enterprises use a SaaS platform to govern AI agents? Most regulated organizations will not route sensitive data access through a third-party SaaS endpoint. Self-hosting the gateways inside your own network perimeter or VPC keeps agent traffic, prompts, and transcripts under your control. Smaller organizations without complex compliance requirements may find SaaS adequate.

How do you prevent AI agents from exceeding their budget? Enforce quotas at the infrastructure level: a fixed token budget, a cap on tool invocations, a maximum runtime, and a limit on sub-agents, with sub-agents drawing from the parent’s pool. When the budget is exhausted the agent stops and requests a human-approved, time-bounded extension.