NewCompare CPU & GPU pricing across AWS, Azure & GCP
Self-hosted sandboxes

Policies and guardrails

How a SandboxPolicy governs a class of sandboxes, how cluster guardrails cap every policy, and why the sandbox namespace is the security boundary.

Policies and guardrails

Two objects control sandboxes on a cluster:

  • a SandboxPolicy governs one class of sandboxes: runtime, image, resources, lifetime, network, warm pool, limits and who may use them. It plays the role for sandboxes that a node policy plays for nodes.
  • SandboxClusterGuardrails is one cluster-wide ceiling that every policy on the cluster must stay within.

You author both in DevZero, on the Sandbox Policies and Sandbox Guardrails pages; a policy belongs to one cluster. The cluster picker on those pages offers only connected clusters. DevZero stores them and applies them to the cluster as read-only objects. If someone edits one in the cluster, DevZero's next push overwrites the edit, and a policy edited in the cluster reports Drifted=True until then.

A policy

apiVersion: sandbox.devzero.io/v1alpha1
kind: SandboxPolicy
metadata:
  name: default                     # immutable, cluster-unique, at most 40 characters
spec:
  displayName: "Default sandboxes"  # free text, shown in the UI
  namespace: devzero-sandbox        # where this policy's sandboxes run
  runtime:
    nodePolicyRef: { name: sandbox-pool }   # a DevZero node policy with Sandbox runtime: gVisor
  image:
    default: public.ecr.aws/devzeroinc/sandbox-base:1
    allowedRegistries: ["public.ecr.aws/devzeroinc/"]
    allowOverride: true
  resources:
    default: { cpu: "1", memory: "2Gi" }
    max:     { cpu: "4", memory: "8Gi" }
  lifecycle:
    defaultTTL: 2h
    maxTTL: 24h
    idleTimeout: 30m
  storage:
    workspace:                      # the /workspace volume
      size: 10Gi
      storageClassName: gp3         # empty: the cluster's default StorageClass
  warmPool:
    size: 2                         # pre-started sandboxes for fast creation
  limits:
    maxSandboxes: 50
  access:
    serviceAccounts: [{ namespace: agent-platform, name: runner }]

The name becomes the name of the objects DevZero creates for the policy and appears in sandbox URLs, so it cannot change. Rename a policy by changing displayName.

Workspace storage

Each sandbox's /workspace is a PersistentVolumeClaim of storage.workspace.size (10Gi by default) on storage.workspace.storageClassName, or on the cluster's default StorageClass when that is empty. The policy condition StorageClassReady reports a named class that does not exist, no default class when none is named (NoDefaultStorageClass), or a class whose cloud CSI driver is not installed (CSIDriverNotInstalled). It does not gate Ready; see Storage.

With no StorageClass that can provision volumes, set storage.workspace.enabled: false (in the API's JSON, "storage": { "workspace": { "enabled": false } }). The sandbox then has no volume: /workspace is a directory on the container's own filesystem, and its files are lost whenever the sandbox's pod is recreated. StorageClassReady reports True, saying the workspace volume is disabled.

Runtime selection in Phase 1

In Phase 1, spec.runtime must name a DevZero node policy that has Sandbox runtime: gVisor turned on (nodePolicyRef). DevZero resolves it to that node policy's RuntimeClass, devzero-gvisor. A policy that sets runtime.runtimeClassName instead is rejected with an error stating that the bring-your-own RuntimeClass form arrives in Phase 2. A node policy without a sandbox runtime leaves the policy RuntimeReady=False with reason NodePolicyNoSandboxRuntime.

Policy status

The dashboard shows each policy's conditions. A policy accepts new sandboxes only while it is Ready, which requires RuntimeReady, WithinGuardrails, QuotaValid, its objects applied, NetworkPolicyEnforced (unless the guardrails allow unenforced operation) and that it is not draining. A claim against a policy that is not ready is refused with PolicyNotReady, and the message names the failing condition, for example NetworkPolicyNotEnforced or ExceedsClusterGuardrails.

The policy page shows the status the cluster last reported and refreshes it on its own: every 5 seconds while the policy is not yet reported or not ready, and every 30 seconds after that, so a later change (a save, or a guardrails change that moves the policy in or out of bounds) still shows up without a reload. SandboxdImageUnreachable=True (the sandboxd injector image cannot be fetched) is a warning only; it does not make the policy not ready, but the sandboxes it breaks fail to start.

Guardrails

A team Admin sets the guardrails in DevZero. The sandbox operator in your cluster checks every policy against them: a policy that exceeds them gets WithinGuardrails=False and Ready=False and is not applied, and the sandbox admission webhooks enforce the policy and the guardrails together on every claim and pod.

Guardrails keep policies within a ceiling your Admin chose and catch mistakes. They do not protect you from DevZero itself: DevZero delivers the guardrails along with the policies, so a compromised control plane could widen both. See Trust and data disclosure for the protections that do hold against DevZero.

GuardrailWhat it capsEnforced by
managedNamespaces (default ["devzero-sandbox"], exact names; patterns are opt-in)the namespaces policies may usepolicy admission and the policy reconciler. An existing namespace is adopted only if you labelled it sandbox.devzero.io/managed=true and it holds no pods other than sandboxes; otherwise the policy reports NamespaceReady=False/NamespaceNotManaged and nothing is applied to it. A pattern never adopts an unlabelled namespace.
allowedRegistriesevery policy's registry list, default image and overrides, and every container in a sandbox podpolicy, claim and pod admission
allowedEnvFromSecrets (a label selector) plus the per-Secret annotation sandbox.devzero.io/exposable-to: "<policy>,…"which Secrets a policy can expose to sandboxes through envFrom: the Secret must match the selector and name the policythe policy reconciler (EnvFromSecretNotAllowed). The sandbox operator cannot write Secrets in sandbox namespaces, so it cannot grant this consent to itself.
allowedRuntimeClassesthe RuntimeClass a policy resolves tothe policy reconciler. The class must be on the list and its handler must be runsc or kata* unless the class is listed by exact name (RuntimeReady=False/RuntimeClassNotSandbox).
allowedClusterServiceNamespaceswhich namespaces' Services sandboxes may reachthe policy reconciler
access.allowedAdminSubjects (an explicit list; empty means none)who a policy may name in access.adminSubjects. system:authenticated, system:unauthenticated, system:serviceaccounts, system:serviceaccounts:* and system:masters are refused even if listed.policy admission and the reconciler (AdminSubjectNotAllowed; no admin Role is created)
maxResources, maxTTLeach policy's resources.max and lifecycle.maxTTLpolicy admission
access.allowedServiceAccountNamespacesthe namespaces of service accounts a policy may grant accessthe policy reconciler (no RoleBinding is created for others)
endpoint.allowRelay, endpoint.allowPublicVisibilitythe endpoint's mode and visibilitythe endpoint reconciler
network.allowUnenforcedNetworkPolicywhether sandboxes may start while the NetworkPolicy canary failsclaim admission

The cluster starts with a default that is permissive where that is safe and explicit where it is not: namespace devzero-sandbox only, any registry, the known sandbox RuntimeClasses (devzero-gvisor, gvisor, kata-*, each handler-checked), no maxResources or maxTTL ceiling (each policy's own resources.max and lifecycle.maxTTL still apply), envFrom only from Secrets labelled sandbox.devzero.io/exposable=true, no admin subjects, no service-account namespaces, public visibility allowed, relay off, and sandboxes blocked while NetworkPolicy is unenforced. With no service-account namespaces listed, a policy's access.serviceAccounts entries get no RoleBinding until your Admin adds their namespaces to access.allowedServiceAccountNamespaces. Your Admin's first save in DevZero replaces the default.

Tightening the guardrails below a running policy does not kill anything: the policy reports WithinGuardrails=False, new sandboxes are refused, existing ones run until they expire, and the dashboard shows which fields are over the limit.

The namespace is the boundary

Every control on a sandbox is anchored to its namespace, never to a label, because anyone who can create a pod can choose its labels. A namespace that holds sandboxes is a sandbox namespace (devzero-sandbox by default). DevZero creates it with the label sandbox.devzero.io/managed=true, Pod Security restricted and service-mesh opt-outs, and never deletes it.

ControlApplies to
Default-deny NetworkPolicyevery pod in the namespace (podSelector: {})
Per-policy egressthe policy's pods (additive; a pod without a policy gets nothing beyond DNS)
Pod admissionevery pod in the namespace. A pod is admitted only if it is owned by a Sandbox and was created by the upstream sandbox controller's service account (DevZero's NetworkPolicy canary pods are the one exception). The webhook also re-checks the runtime class, Pod Security fields, images and resources, and refuses host namespaces and host paths.
pods/exec, attach, portforward, ephemeral containersdenied to everyone except the policy's adminSubjects, with an audit Event
Sandbox lifecycle objects (Sandbox, SandboxTemplate, SandboxWarmPool)created only by the DevZero operator or the upstream controller; their owners and spec cannot be changed by anyone else
Routing to a sandboxby owner references only; labels are never trusted
NetworkPolicy enforcementtested by a canary; sandboxes are blocked while it fails

Put nothing else in a sandbox namespace. Granting anyone any of the following in a sandbox namespace gives them the same power over every sandbox in it as the policy itself: create or patch on pods; pods/exec, pods/attach, pods/portforward or pods/ephemeralcontainers; patch on sandboxes; update on leases; get on Secrets.

A policy cannot capture a namespace you use for something else: DevZero adopts an existing namespace only when you have labelled it sandbox.devzero.io/managed=true and it contains no other pods.

Changing a policy's namespace

spec.namespace defaults to devzero-sandbox and must be one of the guardrails' managedNamespaces. You can change it only when the policy has no sandboxes and no warm pods: set warmPool.size to 0, wait for running sandboxes to end, then change it. Otherwise the change is refused with NamespaceChangeBlocked. DevZero removes the policy's objects from the old namespace and leaves the namespace itself, with its default-deny NetworkPolicy, in place.

Shared or exclusive namespaces

Several policies can share one namespace, or a policy can have a namespace to itself.

Shared namespaceExclusive namespace
Several policies in ityesno, one policy
limits.maxSandboxesenforced per policyenforced
quota (a Kubernetes ResourceQuota, Phase 2)not allowedallowed; the policy reserves the namespace
Network isolation between policiesper-policy egress rules; all pods share the namespace's default denythe namespace's own default deny
Admin rights and RBACeveryone with rights in the namespace reaches every policy's sandboxes in it (the blast radius above)rights in the namespace reach only that policy
Use it whenteams that trust each other share capacity and settingsa team needs a hard resource ceiling or its own RBAC boundary

Limits and quota

limits.maxSandboxesquota (Phase 2)
What it countssandboxes of this policy plus its unclaimed warm podsCPU, memory, storage, volume claims and pods in the namespace
Where it worksany namespace, shared or exclusiveonly a namespace the policy holds exclusively
How it is enforcedthe claim admission webhook (MaxSandboxesExceeded, with current and limit in the message)a Kubernetes ResourceQuota
Under concurrent createsbest effort: two simultaneous creates can both passatomic
Default50none

Warm pods count against maxSandboxes before anyone uses them, so a warm pool of 5 uses 5 of the policy's maxSandboxes while idle. From Phase 2 they count against quota too, and quotas include gVisor's per-pod memory overhead, because Kubernetes counts RuntimeClass overhead in ResourceQuota.

From Phase 2, the exclusive-namespace rule for quota is strict. A policy with quota takes a reservation on its namespace when it is applied; a second policy with quota in the same namespace gets QuotaValid=False/NamespaceReservedByOtherPolicy, and a policy without quota is refused in a namespace another policy has reserved (NamespaceReservedByQuotaPolicy).

Who can use sandboxes

DevZero users, tokens and agents

Anyone DevZero already authenticates (a web session, a personal access token, a team service-account token) is authorized by their role in the team the cluster belongs to:

RoleMay
Viewerlist, view and watch sandboxes and policies
Developereverything a Viewer may, plus create, extend and delete sandboxes and create or revoke sandbox tokens
Admineverything a Developer may, plus author policies, guardrails and endpoints

A policy's access.teams narrows which of the cluster team's members may use that policy; it never widens the role check. Agents and CI should use team service-account tokens.

In-cluster callers

Applications in the cluster create sandboxes with a SandboxClaim and connect with an audience-bound service-account token. Team roles do not apply to them; Kubernetes RBAC and the sandbox webhooks do, at three levels:

  • Namespace, through RBAC. ClusterRole/devzero-sandbox-user lets a service account create, view and connect to claims. It does not grant delete.
  • Policy, through access.serviceAccounts. Empty means RBAC alone decides who may create and connect, whatever bindings you made. A non-empty list is authoritative: DevZero creates RoleBindings for the listed accounts, and anyone not listed is refused even if RBAC would allow them.
  • Sandbox, through the creator. Each in-cluster sandbox records who created it. A service account is recorded in the immutable label sandbox.devzero.io/creator: <namespace>.<service-account>; any other user, for example an IAM identity signed in through an EKS access entry, in the immutable annotation sandbox.devzero.io/creator-user: <username>. By default only the creator may delete the sandbox or change its shutdownTime, and only a service-account creator can connect to it. DevZero gives a service-account creator a Role to delete its own claim; a user creator needs delete on sandboxclaims from RBAC of its own.

Members of system:masters may also create claims, past a policy's access.serviceAccounts list, and delete any claim (the installer default, chart value operator.allowClusterAdminClaims). Other cluster administrators, such as an EKS access entry with the cluster-admin access policy, are not in system:masters: they delete claims they did not create only when the policy lists them in access.adminSubjects.

Three policy fields widen the creator rule:

FieldDefaultEffect
access.sharedWithinPolicyfalseany listed service account may connect to any in-cluster sandbox of the policy; delete stays creator-only
access.adminSubjectsnonethese users, groups or service accounts may connect to and delete any sandbox of the policy, and may use pods/exec; each must be in the guardrails' allowedAdminSubjects
access.allowServiceAccountsOnApiSandboxesfalseservice accounts may connect to sandboxes created through the DevZero API

Deleting a policy: drain

A team Admin deletes a policy from its page in the dashboard (Delete policy) or with the DeleteSandboxPolicy RPC, which takes the policy's ID. Developers and Viewers get forbidden. The policy leaves DevZero's lists at once; deleting it again returns not_found. If the cluster is unreachable, the delete waits and reaches the cluster when it reconnects.

Deleting a policy drains it rather than killing its sandboxes:

  1. The policy reports Draining=True with a deadline (24 hours by default, never later than the policy's maxTTL). New sandboxes are refused with PolicyDraining, and the warm pool scales to zero.
  2. Running sandboxes continue until their lifetime ends, they go idle, or someone deletes them.
  3. At the deadline, DevZero deletes the remaining sandboxes.
  4. DevZero removes the policy's objects. The namespace, its default-deny NetworkPolicy and its CA bundle stay.

While a policy drains, its status in the cluster carries the deadline and the number of sandboxes still running. A policy deleted from DevZero always gets the default deadline. Its name stays reserved: creating a policy with the same name on that cluster is refused with policy_draining until the cluster has confirmed the delete and the longest possible drain (25 hours from that confirmation) has passed.

On this page