Policies and guardrails
How a SandboxPolicy governs a class of sandboxes, how cluster guardrails cap every policy, and why the sandbox namespace is the security boundary.
Policies and guardrails
Two objects control sandboxes on a cluster:
- a
SandboxPolicygoverns one class of sandboxes: runtime, image, resources, lifetime, network, warm pool, limits and who may use them. It plays the role for sandboxes that a node policy plays for nodes. SandboxClusterGuardrailsis one cluster-wide ceiling that every policy on the cluster must stay within.
You author both in DevZero, on the Sandbox Policies and Sandbox Guardrails pages; a policy belongs to one cluster. The cluster picker on those pages offers only connected clusters. DevZero stores them and applies them to the cluster as read-only objects. If someone edits one in the cluster, DevZero's next push overwrites the edit, and a policy edited in the cluster reports Drifted=True until then.
A policy
apiVersion: sandbox.devzero.io/v1alpha1
kind: SandboxPolicy
metadata:
name: default # immutable, cluster-unique, at most 40 characters
spec:
displayName: "Default sandboxes" # free text, shown in the UI
namespace: devzero-sandbox # where this policy's sandboxes run
runtime:
nodePolicyRef: { name: sandbox-pool } # a DevZero node policy with Sandbox runtime: gVisor
image:
default: public.ecr.aws/devzeroinc/sandbox-base:1
allowedRegistries: ["public.ecr.aws/devzeroinc/"]
allowOverride: true
resources:
default: { cpu: "1", memory: "2Gi" }
max: { cpu: "4", memory: "8Gi" }
lifecycle:
defaultTTL: 2h
maxTTL: 24h
idleTimeout: 30m
storage:
workspace: # the /workspace volume
size: 10Gi
storageClassName: gp3 # empty: the cluster's default StorageClass
warmPool:
size: 2 # pre-started sandboxes for fast creation
limits:
maxSandboxes: 50
access:
serviceAccounts: [{ namespace: agent-platform, name: runner }]The name becomes the name of the objects DevZero creates for the policy and appears in sandbox URLs, so it cannot change. Rename a policy by changing displayName.
Workspace storage
Each sandbox's /workspace is a PersistentVolumeClaim of storage.workspace.size (10Gi by default) on storage.workspace.storageClassName, or on the cluster's default StorageClass when that is empty. The policy condition StorageClassReady reports a named class that does not exist, no default class when none is named (NoDefaultStorageClass), or a class whose cloud CSI driver is not installed (CSIDriverNotInstalled). It does not gate Ready; see Storage.
With no StorageClass that can provision volumes, set storage.workspace.enabled: false (in the API's JSON, "storage": { "workspace": { "enabled": false } }). The sandbox then has no volume: /workspace is a directory on the container's own filesystem, and its files are lost whenever the sandbox's pod is recreated. StorageClassReady reports True, saying the workspace volume is disabled.
Runtime selection in Phase 1
In Phase 1, spec.runtime must name a DevZero node policy that has Sandbox runtime: gVisor turned on (nodePolicyRef). DevZero resolves it to that node policy's RuntimeClass, devzero-gvisor. A policy that sets runtime.runtimeClassName instead is rejected with an error stating that the bring-your-own RuntimeClass form arrives in Phase 2. A node policy without a sandbox runtime leaves the policy RuntimeReady=False with reason NodePolicyNoSandboxRuntime.
Policy status
The dashboard shows each policy's conditions. A policy accepts new sandboxes only while it is Ready, which requires RuntimeReady, WithinGuardrails, QuotaValid, its objects applied, NetworkPolicyEnforced (unless the guardrails allow unenforced operation) and that it is not draining. A claim against a policy that is not ready is refused with PolicyNotReady, and the message names the failing condition, for example NetworkPolicyNotEnforced or ExceedsClusterGuardrails.
The policy page shows the status the cluster last reported and refreshes it on its own: every 5 seconds while the policy is not yet reported or not ready, and every 30 seconds after that, so a later change (a save, or a guardrails change that moves the policy in or out of bounds) still shows up without a reload. SandboxdImageUnreachable=True (the sandboxd injector image cannot be fetched) is a warning only; it does not make the policy not ready, but the sandboxes it breaks fail to start.
Guardrails
A team Admin sets the guardrails in DevZero. The sandbox operator in your cluster checks every policy against them: a policy that exceeds them gets WithinGuardrails=False and Ready=False and is not applied, and the sandbox admission webhooks enforce the policy and the guardrails together on every claim and pod.
Guardrails keep policies within a ceiling your Admin chose and catch mistakes. They do not protect you from DevZero itself: DevZero delivers the guardrails along with the policies, so a compromised control plane could widen both. See Trust and data disclosure for the protections that do hold against DevZero.
| Guardrail | What it caps | Enforced by |
|---|---|---|
managedNamespaces (default ["devzero-sandbox"], exact names; patterns are opt-in) | the namespaces policies may use | policy admission and the policy reconciler. An existing namespace is adopted only if you labelled it sandbox.devzero.io/managed=true and it holds no pods other than sandboxes; otherwise the policy reports NamespaceReady=False/NamespaceNotManaged and nothing is applied to it. A pattern never adopts an unlabelled namespace. |
allowedRegistries | every policy's registry list, default image and overrides, and every container in a sandbox pod | policy, claim and pod admission |
allowedEnvFromSecrets (a label selector) plus the per-Secret annotation sandbox.devzero.io/exposable-to: "<policy>,…" | which Secrets a policy can expose to sandboxes through envFrom: the Secret must match the selector and name the policy | the policy reconciler (EnvFromSecretNotAllowed). The sandbox operator cannot write Secrets in sandbox namespaces, so it cannot grant this consent to itself. |
allowedRuntimeClasses | the RuntimeClass a policy resolves to | the policy reconciler. The class must be on the list and its handler must be runsc or kata* unless the class is listed by exact name (RuntimeReady=False/RuntimeClassNotSandbox). |
allowedClusterServiceNamespaces | which namespaces' Services sandboxes may reach | the policy reconciler |
access.allowedAdminSubjects (an explicit list; empty means none) | who a policy may name in access.adminSubjects. system:authenticated, system:unauthenticated, system:serviceaccounts, system:serviceaccounts:* and system:masters are refused even if listed. | policy admission and the reconciler (AdminSubjectNotAllowed; no admin Role is created) |
maxResources, maxTTL | each policy's resources.max and lifecycle.maxTTL | policy admission |
access.allowedServiceAccountNamespaces | the namespaces of service accounts a policy may grant access | the policy reconciler (no RoleBinding is created for others) |
endpoint.allowRelay, endpoint.allowPublicVisibility | the endpoint's mode and visibility | the endpoint reconciler |
network.allowUnenforcedNetworkPolicy | whether sandboxes may start while the NetworkPolicy canary fails | claim admission |
The cluster starts with a default that is permissive where that is safe and explicit where it is not: namespace devzero-sandbox only, any registry, the known sandbox RuntimeClasses (devzero-gvisor, gvisor, kata-*, each handler-checked), no maxResources or maxTTL ceiling (each policy's own resources.max and lifecycle.maxTTL still apply), envFrom only from Secrets labelled sandbox.devzero.io/exposable=true, no admin subjects, no service-account namespaces, public visibility allowed, relay off, and sandboxes blocked while NetworkPolicy is unenforced. With no service-account namespaces listed, a policy's access.serviceAccounts entries get no RoleBinding until your Admin adds their namespaces to access.allowedServiceAccountNamespaces. Your Admin's first save in DevZero replaces the default.
Tightening the guardrails below a running policy does not kill anything: the policy reports WithinGuardrails=False, new sandboxes are refused, existing ones run until they expire, and the dashboard shows which fields are over the limit.
The namespace is the boundary
Every control on a sandbox is anchored to its namespace, never to a label, because anyone who can create a pod can choose its labels. A namespace that holds sandboxes is a sandbox namespace (devzero-sandbox by default). DevZero creates it with the label sandbox.devzero.io/managed=true, Pod Security restricted and service-mesh opt-outs, and never deletes it.
| Control | Applies to |
|---|---|
| Default-deny NetworkPolicy | every pod in the namespace (podSelector: {}) |
| Per-policy egress | the policy's pods (additive; a pod without a policy gets nothing beyond DNS) |
| Pod admission | every pod in the namespace. A pod is admitted only if it is owned by a Sandbox and was created by the upstream sandbox controller's service account (DevZero's NetworkPolicy canary pods are the one exception). The webhook also re-checks the runtime class, Pod Security fields, images and resources, and refuses host namespaces and host paths. |
pods/exec, attach, portforward, ephemeral containers | denied to everyone except the policy's adminSubjects, with an audit Event |
Sandbox lifecycle objects (Sandbox, SandboxTemplate, SandboxWarmPool) | created only by the DevZero operator or the upstream controller; their owners and spec cannot be changed by anyone else |
| Routing to a sandbox | by owner references only; labels are never trusted |
| NetworkPolicy enforcement | tested by a canary; sandboxes are blocked while it fails |
Put nothing else in a sandbox namespace. Granting anyone any of the following in a sandbox namespace gives them the same power over every sandbox in it as the policy itself: create or patch on pods; pods/exec, pods/attach, pods/portforward or pods/ephemeralcontainers; patch on sandboxes; update on leases; get on Secrets.
A policy cannot capture a namespace you use for something else: DevZero adopts an existing namespace only when you have labelled it sandbox.devzero.io/managed=true and it contains no other pods.
Changing a policy's namespace
spec.namespace defaults to devzero-sandbox and must be one of the guardrails' managedNamespaces. You can change it only when the policy has no sandboxes and no warm pods: set warmPool.size to 0, wait for running sandboxes to end, then change it. Otherwise the change is refused with NamespaceChangeBlocked. DevZero removes the policy's objects from the old namespace and leaves the namespace itself, with its default-deny NetworkPolicy, in place.
Shared or exclusive namespaces
Several policies can share one namespace, or a policy can have a namespace to itself.
| Shared namespace | Exclusive namespace | |
|---|---|---|
| Several policies in it | yes | no, one policy |
limits.maxSandboxes | enforced per policy | enforced |
quota (a Kubernetes ResourceQuota, Phase 2) | not allowed | allowed; the policy reserves the namespace |
| Network isolation between policies | per-policy egress rules; all pods share the namespace's default deny | the namespace's own default deny |
| Admin rights and RBAC | everyone with rights in the namespace reaches every policy's sandboxes in it (the blast radius above) | rights in the namespace reach only that policy |
| Use it when | teams that trust each other share capacity and settings | a team needs a hard resource ceiling or its own RBAC boundary |
Limits and quota
limits.maxSandboxes | quota (Phase 2) | |
|---|---|---|
| What it counts | sandboxes of this policy plus its unclaimed warm pods | CPU, memory, storage, volume claims and pods in the namespace |
| Where it works | any namespace, shared or exclusive | only a namespace the policy holds exclusively |
| How it is enforced | the claim admission webhook (MaxSandboxesExceeded, with current and limit in the message) | a Kubernetes ResourceQuota |
| Under concurrent creates | best effort: two simultaneous creates can both pass | atomic |
| Default | 50 | none |
Warm pods count against maxSandboxes before anyone uses them, so a warm pool of 5 uses 5 of the policy's maxSandboxes while idle. From Phase 2 they count against quota too, and quotas include gVisor's per-pod memory overhead, because Kubernetes counts RuntimeClass overhead in ResourceQuota.
From Phase 2, the exclusive-namespace rule for quota is strict. A policy with quota takes a reservation on its namespace when it is applied; a second policy with quota in the same namespace gets QuotaValid=False/NamespaceReservedByOtherPolicy, and a policy without quota is refused in a namespace another policy has reserved (NamespaceReservedByQuotaPolicy).
Who can use sandboxes
DevZero users, tokens and agents
Anyone DevZero already authenticates (a web session, a personal access token, a team service-account token) is authorized by their role in the team the cluster belongs to:
| Role | May |
|---|---|
| Viewer | list, view and watch sandboxes and policies |
| Developer | everything a Viewer may, plus create, extend and delete sandboxes and create or revoke sandbox tokens |
| Admin | everything a Developer may, plus author policies, guardrails and endpoints |
A policy's access.teams narrows which of the cluster team's members may use that policy; it never widens the role check. Agents and CI should use team service-account tokens.
In-cluster callers
Applications in the cluster create sandboxes with a SandboxClaim and connect with an audience-bound service-account token. Team roles do not apply to them; Kubernetes RBAC and the sandbox webhooks do, at three levels:
- Namespace, through RBAC.
ClusterRole/devzero-sandbox-userlets a service account create, view and connect to claims. It does not grant delete. - Policy, through
access.serviceAccounts. Empty means RBAC alone decides who may create and connect, whatever bindings you made. A non-empty list is authoritative: DevZero creates RoleBindings for the listed accounts, and anyone not listed is refused even if RBAC would allow them. - Sandbox, through the creator. Each in-cluster sandbox records who created it. A service account is recorded in the immutable label
sandbox.devzero.io/creator: <namespace>.<service-account>; any other user, for example an IAM identity signed in through an EKS access entry, in the immutable annotationsandbox.devzero.io/creator-user: <username>. By default only the creator may delete the sandbox or change itsshutdownTime, and only a service-account creator can connect to it. DevZero gives a service-account creator a Role to delete its own claim; a user creator needsdeleteonsandboxclaimsfrom RBAC of its own.
Members of system:masters may also create claims, past a policy's access.serviceAccounts list, and delete any claim (the installer default, chart value operator.allowClusterAdminClaims). Other cluster administrators, such as an EKS access entry with the cluster-admin access policy, are not in system:masters: they delete claims they did not create only when the policy lists them in access.adminSubjects.
Three policy fields widen the creator rule:
| Field | Default | Effect |
|---|---|---|
access.sharedWithinPolicy | false | any listed service account may connect to any in-cluster sandbox of the policy; delete stays creator-only |
access.adminSubjects | none | these users, groups or service accounts may connect to and delete any sandbox of the policy, and may use pods/exec; each must be in the guardrails' allowedAdminSubjects |
access.allowServiceAccountsOnApiSandboxes | false | service accounts may connect to sandboxes created through the DevZero API |
Deleting a policy: drain
A team Admin deletes a policy from its page in the dashboard (Delete policy) or with the DeleteSandboxPolicy RPC, which takes the policy's ID. Developers and Viewers get forbidden. The policy leaves DevZero's lists at once; deleting it again returns not_found. If the cluster is unreachable, the delete waits and reaches the cluster when it reconnects.
Deleting a policy drains it rather than killing its sandboxes:
- The policy reports
Draining=Truewith a deadline (24 hours by default, never later than the policy'smaxTTL). New sandboxes are refused withPolicyDraining, and the warm pool scales to zero. - Running sandboxes continue until their lifetime ends, they go idle, or someone deletes them.
- At the deadline, DevZero deletes the remaining sandboxes.
- DevZero removes the policy's objects. The namespace, its default-deny NetworkPolicy and its CA bundle stay.
While a policy drains, its status in the cluster carries the deadline and the number of sandboxes still running. A policy deleted from DevZero always gets the default deadline. Its name stays reserved: creating a policy with the same name on that cluster is refused with policy_draining until the cluster has confirmed the delete and the longest possible drain (25 hours from that confirmation) has passed.
Compatibility
Which Kubernetes versions and which upstream agent-sandbox version each DevZero sandbox release supports, and what the installer refuses.
Exposure modes
How agents outside your cluster reach sandboxes, what each exposure mode sends through DevZero, who holds the DNS name, certificate and key, and how to fix a failed setup.