Agent security: a design for running untrusted code

This post is a design. None of it is built: the broker, sandbox image, egress proxy, token minter, and supervisor do not exist on disk. Part 5 tracks the milestones and Part 15 records what shipped first.

Part 2 described an interactive agent environment. This post covers agents that run unattended: scheduled maintenance agents, code review bots, and ops agents that triage production issues. Such an agent writes code, runs terminal commands, and calls external APIs, so it is a remote code execution engine with network access. If it holds cloud credentials, one prompt injection compromises production. The general threats and the desktop-level defences are in Part 3; this post covers only the agent-specific ones.

Zero-secret architecture

The design is documented in nix/autonomous-agents-design.md (I called it Pattern 2 there, against an earlier design where the agent held its own credentials):

The agent process holds zero secrets. A control plane (MCP broker) on the host holds every credential and mediates every privileged action. The agent can only do what the broker exposes as a narrow, audited tool.

Threat model and mitigations

The two agents

Two agents are planned, each with its own capabilities.

Code agent

Ops agent

Update, 2026-10-15: I now do ticket and queue triage through a retro command instead of an ops agent, so this agent was not built.

The MCP broker: control plane

The broker is designed to expose exactly 7 tools. Each has per-action enforcement:

Hard-blocked everywhere: chat post, email send, DB write, infra mutation, git merge, force-push, push to protected branches.

The design requires every tool call to be appended to a structured audit log: agent ID, tool name + params, timestamp, result (success/failure + truncated response). The broker should refuse any call that bypasses the logger.

LLM inference: session-scoped Bedrock credentials

The most important design decision here is how the sandboxed agent calls the LLM.

Rejected option: pass a long-lived API key through the broker. This would require the broker to proxy every LLM request or trust the agent with a key.

Selected approach: at job start, the host mints short-lived credentials with a scoped AssumeRole session and injects them into the sandbox as the standard AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN variables. I first designed this around a Bedrock API key in AWS_BEARER_TOKEN_BEDROCK, but the code I actually run uses STS credentials, so the design now matches it:

const { Credentials } = await sts.assumeRole({
  RoleArn: "arn:aws:iam::...:role/agent-bedrock-minter",
  RoleSessionName: `agent-${jobId}`,
  Policy: {
    "Version": "2012-10-17",
    "Statement": [{
      Effect: "Allow",
      Action: [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream"
      ],
      Resource: [
        "arn:aws:bedrock:region-1::foundation-model/anthropic.claude-sonnet-4-*"
      ]
    }]
  },
  DurationSeconds: 43200  // 12 hours or job length, whichever shorter
});

Why it is safe: the credentials only authorise the Bedrock runtime. Leaked credentials cannot pivot to the database, logs, queues, or S3. They live for at most 12 hours, or however long the job runs, and the session policy restricts it to specific model ARNs, so the agent can't invoke arbitrary models. Token-cost DoS and prompt exfiltration remain open. CloudWatch billing alarms and model-invocation logging are the planned mitigations.

Isolation layer: gVisor

The sandboxing technology is gVisor: a user-space kernel that intercepts every syscall from the agent process. Unlike Docker containers (which share the host kernel), gVisor provides a second kernel boundary:

docker run --runtime=runsc \
  --rm \
  -v /path/to/repo:/workspace:ro \
  -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e AWS_SESSION_TOKEN \
  agent-image \
  run-task.sh

~/.aws, ~/.ssh, ~/.work, and the host /home/ must not be reachable from inside the sandbox. Egress is denied except to the allowlist. The Bedrock credentials must fail on any non-Bedrock AWS API (s3 ls returns denied). Forbidden tools must be absent or blocked. The checklist below verifies this before go-live.

Verification checklist (before go-live)

□ Confirm ~/.aws, ~/.ssh, ~/.work are NOT reachable from inside sandbox
□ Confirm only the scoped repo bind-mount is visible
□ Confirm egress to arbitrary hosts fails
□ Confirm Bedrock credentials cannot call S3/EC2/RDS
□ Confirm forbidden git commands are blocked
□ Confirm every broker call lands in the audit log

Job lifecycle

The supervisor should enforce this sequence:

1. TRIGGER: Manual / scheduler only
   (No ticket or chat polling, external text is data, never a trigger)

2. SPAWN: Supervisor mints scoped Bedrock credentials
   → starts fresh gVisor sandbox
   → bind-mounts scoped share
   → opens broker socket via bind-mounted Unix domain socket
   → closes all other egress

3. RUN: Agent works within restricted tool manifest
   → LLM calls go directly to Bedrock (no broker bottleneck)
   → All privileged actions go through broker (audited)

4. ARTIFACTS:
   Code agent: auto-commits to feature branch → opens PR (never merges)
   Ops agent:  auto-creates tickets → writes handoff docs to review dir

5. TEARDOWN: Sandbox destroyed
   → Bedrock credentials expire
   → Outputs collected to audit log
   → Ephemeral: nothing to poison next run

The review boundary is the merge step (code) and ticket triage (ops), both human and out-of-band.

Phase 2: Firecracker microVMs

gVisor is Phase 1. The design document describes Phase 2: Firecracker microVMs.

Firecracker is AWS's open-source VM manager (the technology behind Lambda and Fargate). Each agent runs in its own microVM with hardware virtualisation via KVM, stronger isolation than gVisor's user-space kernel. No shared kernel surface at all. Virtio-fs handles shared filesystems; vsock replaces the Unix domain socket for host-guest communication.

When I wrote this, the repo had a Firecracker VM configuration for the Hermes agent (nix/vms/hermes/). It never booted on real hardware, and it has since been removed. The repo now keeps only design notes, including nix/autonomous-agents-firecracker-handoff.md. Phase 2 is untested.

Deployment: NixOS systemd services

The plan is to declare the whole system in Nix. The file below does not exist:

# Planned in nix/home/autonomous-agents.nix (not created)
services.agent-supervisor = {
  enable = true;
  agents = {
    code = {
      sandbox = "runsc";          # or "firecracker" in Phase 2
      repo-bind = "/home/developer/repos/work-project";
      schedule = "daily 06:00";
    };
    ops = {
      sandbox = "runsc";
      schedule = "daily 07:00";
    };
  };
};

The supervisor would run as a systemd --user service, with each sandbox spawned as a transient systemd scope for resource accounting.

What the design depends on

Nix would make the sandbox image, broker, and systemd units reproducible across machines. The broker speaks MCP, which the agent stack from Part 2 already uses. Secrets handling and audit logging would reuse the sops-nix setup from Part 3.

Part 5 turns this design into a roadmap.