# How Should Teams Design Runtime Agent Security Architecture in 2026?

infonesia.fyi · September 24, 2026

> Direct Answer: Treat the Agent as an Untrusted Program Runtime agent security architecture is the set of controls placed around an AI agent while it is...

## Direct Answer: Treat the Agent as an Untrusted Program

Runtime agent security architecture is the set of controls placed around an AI agent while it is running, rather than only reviewing prompts, model outputs, or source code before deployment. The working assumption should be that an agent may misinterpret instructions, follow hostile content, select the wrong tool, leak sensitive data, or take an action its operator never intended. A defensible design therefore gives the model limited authority, routes every consequential action through policy enforcement, isolates execution, controls network and data access, and produces evidence that a human or auditor can inspect. Recent projects discussed in the research material, including Cupcake, LawClaw, and runtime-security tools for injection and tool abuse, point in the same direction, but they represent different approaches rather than interchangeable products.

**Also worth reading:** [How Do Enterprise Teams Conduct Rigorous RAG Security Testing in Production Environments?](https://infonesia.fyi/knowledge/how_do_enterprise_teams_conduct_rigorous_rag_security_testing_in_production_environments.php) · [What Are the Best AI Agent Security Practices for Indonesian and SEA Enterprises in 2026?](https://infonesia.fyi/knowledge/what_are_the_best_ai_agent_security_practices_for_indonesian_and_sea_enterprises_in_2026.php) · [How Can Enterprise Teams Design a Resilient AI Knowledge Operations Strategy in Southeast Asia?](https://infonesia.fyi/knowledge/how_can_enterprise_teams_design_a_resilient_ai_knowledge_operations_strategy_in_southeast_asia.php)

For Indonesian and Southeast Asian enterprise teams, this means separating four concepts that are often wrongly bundled together: model safety, application security, infrastructure security, and operational governance. A model may be difficult to predict, while an agent platform is an ordinary distributed application with non-deterministic decisions. The platform should remain verifiable even when the model is not. A useful target is to mediate 100% of side-effecting tool calls, deny direct internet egress by default, scope credentials to a single task, and retain tamper-resistant logs for at least 90 days for many regulated deployments. Runtime controls are not a replacement for training, prompt review, or conventional vulnerability testing; they are the layer that limits damage when those controls fail.

## The Security Architecture an Agent Actually Needs

A practical agent runtime has at least eight linked layers. The first is identity: a human owner, a workload identity, and a short-lived session must be distinguishable, and the model itself should never hold unrestricted user credentials. The second is a policy decision point, preferably using deny-by-default rules based on the user, agent, task, tool, resource, and risk level. The third is a tool broker that replaces unrestricted function calling with a narrow interface capable of validating arguments and returning only the data a task needs. The fourth is a context assembly service that labels instructions by provenance, separates trusted system rules from untrusted documents, and removes data that the current task does not require.

The remaining layers concern execution and observation. Sandboxing keeps code tools inside short-lived containers, microVMs, or similarly isolated environments with CPU, memory, process, and time limits. A secrets broker injects credentials just before approved operations and revokes them afterward. Egress controls restrict destinations, protocols, upload sizes, and data classification, while audit records capture prompts, policy decisions, tool arguments, outputs, and approvals. Kernel-level mechanisms such as eBPF may help observe runtime behavior, but host-level monitoring does not by itself tell you whether a particular database export was permitted by business policy. Meta's reported use of a kernel-level sentinel for Muse and industry reporting about runtime gateways from vendors such as Okta are therefore relevant signals, not evidence that one control solves the entire problem.

A common logical request flow is: receive a task, authenticate the session, retrieve the minimum necessary context, propose an action, evaluate policy, obtain approval when required, execute through a scoped tool, inspect the result, and write an audit event. The agent should not be able to bypass the broker by calling a database driver, shell, browser, or cloud API directly. If every step is mediated, teams can change policy without retraining the model, revoke a compromised session quickly, and explain unusual behavior with evidence rather than inference.

## Policy Enforcement and Tool Governance

Tool governance is where runtime agent security becomes operational. A tool such as “search the web,” “send email,” “run SQL,” or “deploy code” is not one capability; it is a family of actions with different risks. Searching public documentation may be low risk, while reading a customer table, changing a billing record, or transferring funds requires stronger conditions. A policy engine such as Open Policy Agent, as used in the Cupcake project described in the research material, can evaluate structured decisions outside the model, making controls more deterministic and testable. LawClaw’s constitutional-governance concept similarly suggests that permissible behavior can be expressed as explicit rules, although the practical value depends on rule quality, context accuracy, and enforcement outside the prompt.

Policies should consider at least six dimensions: who initiated the request, which agent identity is acting, what data the action touches, how sensitive that data is, whether the destination is approved, and how irreversible the result is. Thresholds should be measurable. For example, read-only retrieval of public data might be allowed automatically, access to internal records might require an approved service account, and any external transfer above 5 MB or containing regulated fields might require human confirmation. Destructive actions such as deleting more than 100 records, changing production infrastructure, or sending messages to more than 10 external recipients should be blocked until an authorized person approves them. These numbers are policy defaults, not universal standards, and should be adjusted through risk assessment and testing.

The system must also handle sequences of individually harmless actions. Twenty permitted reads can become a serious exfiltration path when combined, just as repeated small refunds can create a larger financial loss. Runtime analysis can track accumulated data volume, destination diversity, and action frequency within a session. Consecutive tool errors, retries, privilege changes, or access to new data classes should raise suspicion and trigger session suspension. A policy engine that evaluates only one function call at a time will miss many multi-step attacks, so teams need session context without allowing the model itself to rewrite the rules.

## Isolation, Sandboxing, and Egress Control

Code-execution agents require isolation because the tool often runs commands assembled from natural language. A container is a reasonable minimum for low-risk workloads, but it is not a magical boundary, particularly where kernels, mounts, credentials, or container privileges are shared. High-risk execution should use a microVM, a disposable sandbox, a separate account, and a fresh filesystem. Production hosts should never be the default execution environment. A practical baseline is a 30-minute maximum session, 1 GB of memory, 1 vCPU, and no inbound network access unless the tool is specifically designed for it; stronger isolation may be needed for untrusted code.

Network egress deserves separate treatment. An agent with unrestricted HTTPS access can send data to an arbitrary service, and a content filter alone may not detect encoded or fragmented transfers. Teams should use an egress proxy or gateway with destination allowlists, DNS controls, upload limits, and classification-aware inspection. If an agent can browse the web, it should receive reduced credentials and sanitized responses. Internal databases should be reached through purpose-built services or read-only replicas rather than broad production credentials. AWS, CrowdStrike, and Okta are among the organizations named in the research context as working around agent runtime gateways, but the existence of a gateway product does not remove the need for cloud IAM, endpoint protection, or network segmentation.

Kernel-level monitoring can add visibility into process execution, file access, and network behavior. The cited Datadog discussion of eBPF dates to February 2021, before the current generation of coding and tool-using agents, so it should be treated as infrastructure background rather than an agent-specific security standard. It can reveal suspicious behavior, but interpretation still requires workload context. Teams should not confuse continuous monitoring with prevention: a kernel sentinel may observe an unauthorized call successfully completing unless an enforcement path blocks it.

## Identity, Secrets, Memory, and Context Security

Agent security often fails because authority is attached too broadly. Instead of giving a model a personal access token, issue a task-scoped identity with a lifetime of 5 to 15 minutes and permissions limited to named resources. A sales agent permitted to read a deal may not need permission to export its contact history, while a support agent may need customer context but not internal payroll data. Secrets should be brokered, encrypted, and injected only for the specific operation, with automatic rotation after use or session termination. Long-lived API keys stored in prompts, vector databases, or chat histories create persistent risk even if the original tool policy is sound.

Memory deserves the same scrutiny as any production datastore. Long-term memory can become a hidden channel for poisoned instructions or sensitive information. Store provenance, tenant, creation time, sensitivity, and permitted uses with every item, and filter retrieval by user and task. A memory record should not automatically regain authority because it was written by an earlier agent session. Teams can reduce exposure by retaining conversational memory for 30 days, audit records for 90 days or longer, and regulated content according to applicable legal and contractual requirements. These are starting points for policy design, not universal compliance periods.

Prompt injection remains difficult to eliminate, so architecture must assume it will occur. External pages, PDFs, emails, and shared documents should be treated as data with untrusted instructions, not as system messages. Strip active content where possible, isolate browsing sessions, label provenance, and prevent retrieved text from changing the policy engine. Two models can disagree about whether content is malicious; only a trusted enforcement layer can decide whether an action is allowed. This is why “instruction hierarchy” in the prompt should be supported by technical boundaries outside it.

## Observability, Incident Response, and Evidence

Runtime security is incomplete if teams cannot reconstruct what happened. Logs should include the request owner, agent version, model version, tool name, normalized arguments, data classification, policy version, decision, approver, result status, and correlation ID. Record both permitted and denied actions, because repeated denials may reveal attack sequences or broken agent behavior. Logs should be written to append-only storage with restricted deletion rights, synchronized time, and tenant separation. High-risk actions may need stronger evidence than routine reads, including hashes of input and output so investigators can detect later modification.

Detection should focus on behavior rather than dramatic wording alone. Warning signals include a sudden jump in tool calls, access across multiple customers, repeated access to credential-related files, attempts to contact new external domains, policy-denial loops, unusual token consumption, and activity outside the agent’s normal working hours. Because model outputs vary, fixed keyword alerts will produce many false positives. A practical pilot might begin with 20 to 30 baseline scenarios, measure precision and false-positive rates, and tune thresholds before enabling automated suspension. Human analysts still need playbooks for session termination, credential revocation, data-access review, customer notification, and recovery.

There is also an evidence problem for regulated sectors in Indonesia and SEA. Data residency, sector rules, contractual obligations, and cross-border transfer requirements can differ by country and industry. A control that works technically may still be unacceptable if prompts or logs leave the approved jurisdiction. Before production deployment, map every data store, model provider, tool endpoint, and log destination, then assign an owner and retention rule. Runtime security can enforce some of those decisions, but it cannot decide whether a legal interpretation is correct.

## Practical Implementation Plan for Enterprise Teams

Teams should begin with a bounded workflow rather than an “AI employee” with broad access. Select one process, such as researching a public product category or drafting an internal report, and define the smallest set of tools required. Document the expected inputs, outputs, destinations, data classes, and prohibited actions. Build a threat model that includes direct misuse, compromised tools, malicious documents, stolen credentials, excessive tool permissions, multi-step exfiltration, and model provider failure. Assign risk owners in engineering, security, legal, and the business unit.

Next, create a policy test suite with at least 50 cases before connecting real systems. Include 20 cases that should be allowed, 20 that should be denied, and 10 that require approval. Record the expected policy decision, affected tool, and remediation. Run these tests on every policy, prompt, model, and tool-schema change, because a harmless-looking schema edit can alter security behavior. During a limited pilot, use synthetic or masked data, cap the agent at 50 tool calls per task, and require human review for external communication or production changes. Expand access only when the observed false-positive rate and incident response process are acceptable.

The rollout sequence should be observe, constrain, approve, then automate. In observe mode, record proposed actions and compare them with expected behavior. In constrain mode, use read-only credentials, allowlisted tools, sandboxed execution, and blocked egress. In approve mode, require human confirmation for medium-risk actions. Only mature workflows should receive a narrow degree of automation. A reasonable target for the first 90 days is 100% mediation of tool calls, 0 production credentials directly available to model code, and at least 95% of high-risk actions linked to an auditable decision. These are internal targets, not published standards.

## Alternatives, Trade-offs, and Cost

There is no single product category that replaces the entire architecture. A policy engine offers deterministic decisions but needs accurate context and strong enforcement. A gateway can centralize tool access and telemetry but may not provide deep workload isolation. A sandbox reduces code risk but does not stop a legitimate, approved tool from disclosing the wrong data. eBPF or endpoint monitoring improves detection but may not prevent an action. Model-provider safeguards help with harmful output but do not govern every downstream API call.

| Feature | Central policy and tool broker | Sandboxed isolated execution | Full enterprise governance platform |
| --- | --- | --- | --- |
| Primary strength | Deterministic allow, deny, and approval rules | Limits damage from code and shell tools | Joins identity, audit, policy, incident workflows, and reporting |
| Typical implementation time | 2–8 weeks for a bounded pilot | 4–12 weeks, depending on infrastructure and compliance | 3–9 months across multiple business units |
| Best fit | Teams standardizing agent tools and permissions | Coding, research, and document-processing agents | Regulated or multi-tenant organizations with formal accountability |
| Common weakness | Rules fail when context or provenance is poor | Isolation does not prevent misuse of approved business tools | Higher cost, maintenance, and integration burden |
| Evidence quality | Strong per-decision policy records | Execution and network evidence | Broad, correlated, tenant-aware audit history |

Open-source components can reduce software licensing costs, but operational cost remains. A small pilot may cost roughly US$5,000 to US$25,000 in engineering and security work, while an enterprise rollout can range from US$50,000 to US$500,000 or more after integration, testing, compliance, and staffing. Gateway, policy, sandbox, and observability products may add subscription fees per user, session, protected workload, or million events; pricing is rarely comparable without a defined unit. Managed services can lower implementation effort but may create data-residency concerns. The key cost question is not whether a runtime product is “cheap,” but whether it reduces the number and severity of incidents that would otherwise require manual investigation.

## Common Mistakes and When to Act

The most common mistake is treating prompt instructions as access control. A system message saying “never reveal secrets” is useful behavioral guidance, but it is not a boundary around a shell, database, or browser. The second mistake is allowing every agent to inherit a human’s permissions because it “needs context.” The third is enabling unrestricted internet access to solve a small integration problem. The fourth is retaining every prompt and document in a long-term memory store without provenance or deletion rules. The fifth is buying a security dashboard before defining which actions require human approval.

Teams should act immediately when an agent can modify production data, execute untrusted code, access regulated information, communicate externally without review, or hold credentials shared across customers. If those conditions are present, restrict access first, revoke exposed tokens second, and preserve logs third. For low-risk internal research, a measured pilot may be appropriate, but it should still use a dedicated identity, an allowlisted tool set, and a session time limit. Waiting for a formal agent-security program is not sensible when a reachable production endpoint already has broad authority.

A 30-day remediation plan can be built around four measurable gates: identify every tool and credential, block direct side-effecting calls, test 50 policy scenarios, and rehearse credential revocation and session termination. By day 90, the organization should know which workflows remain manual, which actions are fully automated, and what evidence supports each claim. The most credible architecture is not the one with the most security branding; it is the one where every consequential action has a named owner, a deterministic decision, a narrow permission, and a recoverable record.

The architecture is still evolving. Research references dated up to 25 September 2026 should be checked against current vendor documentation, because agent platforms, regulations, and product capabilities change quickly. Organizations should treat reported examples from Meta, NVIDIA, Okta, AWS, CrowdStrike, and open-source projects as directional evidence rather than independent assurance for their own deployment. The durable principle is simpler: model intelligence should request action, while trusted software decides whether that action can happen and records what happened afterward.

## Quick answers

### What is runtime agent security architecture?

It is the set of identity, policy, isolation, data, network, and monitoring controls enforced while an AI agent operates. It protects the execution path around the model, including tools, credentials, memory, and external systems. The goal is to limit impact when the model is wrong or manipulated.

### Is a sandbox enough to secure a coding agent?

No. A sandbox can reduce damage from untrusted code, but an agent inside the sandbox may still misuse an approved API, read sensitive files, or exfiltrate data through an allowed network route. Combine isolation with least-privilege credentials, policy-checked tools, egress controls, and audit logging.

### How should teams control high-risk agent actions?

Start with deny-by-default rules and route every side-effecting tool call through a trusted broker. Require human approval for external messages, production changes, regulated-data access, financial operations, or large data transfers. Set concrete thresholds, such as 5 MB or 10 recipients, and revise them through testing.

### Does runtime security replace prompt-injection defenses?

No. Prompt-injection defenses reduce the chance that hostile content changes agent behavior, while runtime controls limit the consequences if the injection succeeds. Use provenance-aware context, instruction separation, scoped permissions, and deterministic enforcement outside the model rather than relying on a single safeguard.

### What is a reasonable starting budget for an enterprise pilot?

A bounded pilot often costs about US$5,000 to US$25,000 in engineering, security testing, and integration work, although cloud usage and managed services can change that range. A broader regulated deployment may require US$50,000 to US$500,000 or more. Price the control and operational model, not only the software subscription.

Canonical: https://infonesia.fyi/knowledge/how_should_teams_design_runtime_agent_security_architecture_in_2026.php
Markdown: https://infonesia.fyi/knowledge/how_should_teams_design_runtime_agent_security_architecture_in_2026.php/index.md