SPIRE workload identity on Kubernetes
This guide sets up SPIFFE identities for Edera pods with a standard SPIRE server and the SPIRE Kubernetes registrar (spire-controller-manager). At the end, each Edera pod gets its registrar-written identity in one or both of two ways:
- Workload API. The pod’s containers fetch SVIDs from a SPIRE agent socket.
- Mounted identity. Edera writes the SVID, its key, and the trust bundle into the container as files, for workloads that cannot speak the Workload API.
The same ClusterSPIFFEID serves both. Nothing is registered per workload.
For what SPIFFE and SPIRE are, and how Edera runs a SPIRE agent in each zone, see Workload identity with SPIFFE and SPIRE.
How it works
Every Edera pod runs in its own zone, and each zone runs its own SPIRE agent. The agent does not attest with a Kubernetes token. It presents evidence: a statement about the zone that the Edera daemon on the node signs with a node key.
Each node gets its node key from SPIRE. A small DaemonSet, the node enroller, proves which node it runs on with its service account token, and SPIRE issues it an identity for that node. The enroller writes that identity’s key where the daemon reads it, and renews it before it expires.
The SPIRE server’s Edera plugin checks that the evidence is signed by the node the pod runs on, checks the pod against the Kubernetes API, and names the agent after the pod:
spiffe://<trust domain>/spire/agent/edera-hypervisor/<node name>/pod/<pod uid>The registrar renders each Edera pod’s entries under that agent, so a zone can only ever hold identities for its own pod.
Prerequisites
- A Kubernetes cluster with Edera installed on the nodes that run Edera pods. See the installation guides.
- The API server runs with the Node authorizer and the
NodeRestrictionadmission plugin. Most distributions turn both on by default. kubectl,helm, anddocker(orcrane) on your workstation.- Shell access to each Edera node, to configure the daemon.
- Outbound access from the cluster to
ghcr.io.
This guide was tested with these versions:
| Component | Version |
|---|---|
spire-crds chart | 0.6.1 |
spire chart | 0.30.2 |
| Edera spire-controller-manager | 0.7.1-edera.0 (ghcr.io/edera-dev/spire-controller-manager@sha256:4b0bb917ff67b00fc8df87d05660b59e9d4d70d1e6f00f125727896047f3568d) |
| Edera attestor plugins | v0.0.13 (ghcr.io/edera-dev/edera-attestor-plugins@sha256:deba8624d3d35ec66db8b3b79f9459cba1d4601d0eebfcaf23ea40c212e2f864) |
edera-node-enroller chart | 1.2.0 |
The registrar image is Edera’s fork of spire-controller-manager. The standard upstream spire charts and CRDs can be used with it unchanged.
Set these variables for the rest of the guide:
export TRUST_DOMAIN=example.org # your SPIFFE trust domain
export CLUSTER_NAME=my-cluster # the registrar's cluster name
export NAMESPACE=spire-system
export REGISTRAR_CLASS=spire # the registrar's class name
export SPIRE_SERVER=spire-server.${NAMESPACE}:443 # the SPIRE server that enrollers reach, as host:port
export PLUGINS_VERSION=v0.0.13
export PLUGINS_IMAGE=ghcr.io/edera-dev/edera-attestor-plugins:${PLUGINS_VERSION}
export NODE_ARCH=linux/amd64Step 1: Check the RuntimeClass
The registrar decides which parent an entry gets from the pod’s node selector, so every Edera pod needs runtime: edera in its node selector. The standard edera RuntimeClass adds it. If you have not applied it and labeled the Edera nodes, follow Apply the Edera RuntimeClass and Label nodes for Edera workloads.
Check that the RuntimeClass has the node selector:
kubectl get runtimeclass edera -o jsonpath='{.scheduling.nodeSelector}'
# {"runtime":"edera"}A pod’s node selector cannot change after the pod is scheduled, which is why the registrar uses it rather than a label or an annotation.
Step 2: Install SPIRE
Create the namespace, and install the standard CRDs:
kubectl create namespace "$NAMESPACE"
helm upgrade --install spire-crds spire-crds \
--repo https://spiffe.github.io/helm-charts-hardened --version 0.6.1 \
--namespace "$NAMESPACE" --waitThe chart requires the checksum of the server plugin binary. Take it from the plugin image:
docker pull --platform $NODE_ARCH "$PLUGINS_IMAGE"
id=$(docker create --platform $NODE_ARCH --entrypoint /bin/true "$PLUGINS_IMAGE")
docker cp "$id:/var/lib/edera/protect/attestor-plugins/edera-server-nodeattestor" .
docker rm "$id"
export PLUGIN_SHA256=$(sha256sum edera-server-nodeattestor | cut -d' ' -f1)
# Or, with [crane](https://github.com/google/go-containerregistry/blob/main/cmd/crane/README.md)
export PLUGIN_SHA256=$(crane export --platform linux/amd64 "$PLUGINS_IMAGE" - \
| tar -xO var/lib/edera/protect/attestor-plugins/edera-server-nodeattestor \
| sha256sum | cut -d' ' -f1)Write the chart values. The server loads the Edera plugin twice: as edera-hypervisor, which attests zones, and as edera-node, which enrolls nodes.
cat > spire-values.yaml <<EOF
global:
spire:
trustDomain: ${TRUST_DOMAIN}
clusterName: ${CLUSTER_NAME}
spire-server:
controllerManager:
enabled: true
className: ${REGISTRAR_CLASS}
image:
registry: ghcr.io
repository: edera-dev/spire-controller-manager
tag: "0.7.1-edera.0"
parentIDTemplate: '{{ if eq (index .PodSpec.NodeSelector "runtime") "edera" }}spiffe://{{ .TrustDomain }}/spire/agent/edera-hypervisor/{{ .NodeMeta.Name }}/pod/{{ .PodMeta.UID }}{{ else }}spiffe://{{ .TrustDomain }}/spire/agent/k8s_psat/{{ .ClusterName }}/{{ .NodeMeta.UID }}{{ end }}'
customPlugins:
nodeAttestor:
edera-hypervisor:
plugin_cmd: /var/lib/edera/protect/attestor-plugins/edera-server-nodeattestor
plugin_checksum: ${PLUGIN_SHA256}
image:
registry: ghcr.io
repository: edera-dev/edera-attestor-plugins
tag: "${PLUGINS_VERSION}"
plugin_data:
mode: evidence
node_identity: spire
edera-node:
plugin_cmd: /var/lib/edera/protect/attestor-plugins/edera-server-nodeattestor
plugin_checksum: ${PLUGIN_SHA256}
image:
registry: ghcr.io
repository: edera-dev/edera-attestor-plugins
tag: "${PLUGINS_VERSION}"
plugin_data:
mode: enrollment
service_account_allow_list: ["${NAMESPACE}:edera-node-enroller"]
EOFThe parent ID template puts an Edera pod’s entries under that pod’s Edera agent, and every other pod keeps the chart’s normal node agent.
Leave the chart’s PSAT node attestor enabled, which is its default. It also grants the SPIRE server’s service account what the Edera plugin needs: get on pods and nodes, and create on token reviews.
Keep spire-agent and spiffe-csi-driver enabled if non-Edera pods in the cluster need identities. Edera pods use neither.
Install:
helm upgrade --install spire spire \
--repo https://spiffe.github.io/helm-charts-hardened --version 0.30.2 \
--namespace "$NAMESPACE" --values spire-values.yaml --wait --timeout 10mStep 3: Let each zone fetch mounted identities
Edera writes a mounted identity through the SPIRE agent’s Delegated Identity API, which needs a delegate identity for the zone’s own agent. One cluster-wide ClusterSPIFFEID issues it for every pod:
kubectl apply -f - <<EOF
apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterSPIFFEID
metadata:
name: edera-zone-agent
annotations:
spire-controller-manager.edera.dev/additive: "true"
spec:
className: ${REGISTRAR_CLASS}
spiffeIDTemplate: "spiffe://{{ .TrustDomain }}/edera/pod/{{ .PodMeta.UID }}/zone-agent"
workloadSelectorTemplates:
- "edera:principal:zone-agent"
EOFKeep the annotation. The chart’s default ClusterSPIFFEID is a fallback, and any other ClusterSPIFFEID that matches a pod suppresses fallbacks. Without the annotation, this object takes away every pod’s default identity. The annotation only works with the Edera registrar image.
Skip this step if no workload uses mounted identities.
Step 4: Point the Edera daemon on each node at the server
Get the server’s address:
kubectl -n "$NAMESPACE" get service spire-server -o jsonpath='{.spec.clusterIP}'On each Edera node, add this to /var/lib/edera/protect/daemon.toml:
[features]
spire-v0 = "enabled"
[spire]
trust-domain = "example.org" # the same as TRUST_DOMAIN
server-address = "10.96.0.123" # the spire-server ClusterIP
server-port = 443
trust-bundle-path = "/var/lib/edera/protect/spire/trust/server-bundle.pem"
node-key-path = "/var/lib/edera/protect/spire/node/node.key"
node-certificate-path = "/var/lib/edera/protect/spire/node/node.crt"
# evidence-lifetime-seconds = 300 # default 300, from 1 to 86400Then restart the daemon:
sudo systemctl restart protect-daemonThe bundle and the node key do not exist yet. The chart in the next step creates both. Until then, the daemon starts each Edera pod’s zone but holds its SPIRE agent back.
See the daemon.toml reference for every [spire] option.
Step 5: Enroll the nodes
Install the node enroller chart into the SPIRE namespace:
helm upgrade --install edera-node-enroller \
oci://ghcr.io/edera-dev/charts/edera-node-enroller --version 1.2.0 \
--namespace "$NAMESPACE" \
--set trustDomain="$TRUST_DOMAIN" \
--set serverAddress="$SPIRE_SERVER" \
--set image.tag="$PLUGINS_VERSION"The chart runs a DaemonSet on the nodes labeled runtime: edera. Each pod does two things:
- Its init container copies the server’s bundle to the node. It waits until the SPIRE server has published its bundle to the
spire-bundleConfigMap, writes it as the daemon’strust-bundle-path, and exits. It holds no token and no key. - The enroller enrolls the node. It writes
node.keyandnode.crt, and renews them at half their lifetime. The daemon picks up each new pair with no restart. New nodes enroll as soon as the DaemonSet reaches them.
Expected Result: every Edera node is listed as an enrolled agent.
kubectl -n "$NAMESPACE" logs daemonset/edera-node-enroller
kubectl -n "$NAMESPACE" exec spire-server-0 -c spire-server -- \
/opt/spire/bin/spire-server agent list
# ... spiffe://example.org/spire/agent/edera-node/<node uid>Chart values
| Value | Default | Meaning |
|---|---|---|
trustDomain | Required. The same as TRUST_DOMAIN. | |
serverAddress | spire-server.<namespace>:443 | The SPIRE server, as host:port. |
serviceAccount.name | edera-node-enroller | Must be the entry in the edera-node attestor’s service_account_allow_list (step 2). |
bundle.configMap, bundle.key | spire-bundle, bundle.spiffe | The chart’s bundle ConfigMap. Change them if you set global.spire.bundleConfigMap, or bundlePublisher.k8sConfigMap.format: pem (key bundle.crt). |
hostPaths.nodeKeyDirectory, hostPaths.trustDirectory | /var/lib/edera/protect/spire/node, /var/lib/edera/protect/spire/trust | Must match the paths in daemon.toml (step 4). |
nodeSelector, tolerations | runtime: edera, none | Where the enroller runs. Keep it to Edera nodes. |
Properties to keep
- The service account is only for the enroller, and only it is named in
service_account_allow_list. Never add it to the chart’s PSAT allow list (nodeAttestor.k8sPSAT.serviceAccountAllowList). If you do, the enroller can take the node’s standard agent ID, and its key can fetch every non-Edera pod identity on the node. - The token automount is off, and the projected token’s audience is
edera-node. It is not the standard agents’spire-server, so a token for one attestor never passes the other. - The enroller container mounts the trust directory read-only, so it cannot change the bundle that the daemon and every new zone trust. Only the init container writes it. No other host path is mounted, and no port is exposed.
- No container has a Linux capability, and the pod sets no
runtimeClassName. The server refuses an enroller pod that sets one.
Step 6: Run a workload
Ask for an identity with a pod annotation. Use either or both:
| Annotation | Effect |
|---|---|
dev.edera/spire-access: "true" | The containers get the SPIRE agent’s Workload API socket at /shared/zone/sockets/agent.sock. |
dev.edera/spire-identity: "true" | Edera writes the pod’s SVIDs into each container at /run/spire/identity/: svid.0.pem, svid.0.key, and bundle.<trust domain>.pem, and keeps them fresh. |
apiVersion: v1
kind: Pod
metadata:
name: hello-spiffe
annotations:
dev.edera/spire-access: "true"
dev.edera/spire-identity: "true"
spec:
runtimeClassName: edera
containers:
- name: app
image: ghcr.io/spiffe/spire-agent:1.15.3
command: ["/opt/spire/bin/spire-agent", "api", "watch",
"-socketPath", "/shared/zone/sockets/agent.sock"]With the chart’s default ClusterSPIFFEID, this pod gets spiffe://example.org/ns/default/sa/default by either path.
For a SPIFFE library, set SPIFFE_ENDPOINT_SOCKET=unix:///shared/zone/sockets/agent.sock.
Verification
The pod’s zone attests as its own agent:
kubectl -n "$NAMESPACE" exec spire-server-0 -c spire-server -- \
/opt/spire/bin/spire-server agent list
# ... spiffe://example.org/spire/agent/edera-hypervisor/<node>/pod/<pod uid>The registrar wrote the pod’s entry under that agent:
POD_UID=$(kubectl get pod hello-spiffe -o jsonpath='{.metadata.uid}')
kubectl -n "$NAMESPACE" exec spire-server-0 -c spire-server -- \
/opt/spire/bin/spire-server entry show \
-parentID "spiffe://$TRUST_DOMAIN/spire/agent/edera-hypervisor/<node>/pod/$POD_UID"The workload receives it both ways:
kubectl exec hello-spiffe -- /opt/spire/bin/spire-agent api fetch x509 \
-socketPath /shared/zone/sockets/agent.sock
kubectl logs hello-spiffe # the api watch outputFor a container with a shell, cat /run/spire/identity/svid.0.pem shows the same SPIFFE ID in the mounted certificate.
Scoping identities
Write ClusterSPIFFEID objects as for any SPIRE deployment. The k8s: selectors below have exactly the meaning the upstream Kubernetes workload attestor gives them, so existing ClusterSPIFFEID objects work for Edera pods unchanged.
| Selector | Meaning |
|---|---|
k8s:pod-uid, k8s:ns, k8s:pod-name | The pod |
k8s:container-name | The container |
k8s:node-name | The node |
k8s:container-image | The container’s image and imageID from the pod status. Under CRI-O, only the image form. |
k8s:sa | The pod’s service account |
k8s:pod-image-count, k8s:pod-init-image-count | How many containers and init containers the pod spec lists. Left out once the pod has run an ephemeral container, so an entry that needs them stops matching. |
Two upstream selectors have an Edera variant instead, and some are not available:
| Instead of | Use | Difference |
|---|---|---|
k8s:pod-image, k8s:pod-init-image | edera:k8s-pod-image | The images of every container the pod has run, including init and ephemeral containers, and ones that exited |
k8s:pod-label, k8s:pod-owner, k8s:pod-owner-uid, k8s:ns-label | Not available | An entry that needs them matches nothing |
This example gives one identity to the api container of app: payments pods, only under the pod’s own service account. It never matches a pod that has run a container its spec does not list, such as an ephemeral debug container:
apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterSPIFFEID
metadata:
name: payments-api
spec:
className: spire
podSelector:
matchLabels:
app: payments
spiffeIDTemplate: "spiffe://{{ .TrustDomain }}/payments/api"
workloadSelectorTemplates:
- "k8s:container-name:api"
- "k8s:sa:{{ .PodSpec.ServiceAccountName }}"
- "k8s:pod-image-count:{{ len .PodSpec.Containers }}"The edera: selectors also work, for entries written by hand.
Require signed images
An Edera pod can get its identity only when its image has a valid cosign signature from an identity that you trust. This works like the sigstore support of the upstream Kubernetes workload attestor: the same options, the same selectors, and the same failure behavior. The SPIRE agent in each zone checks the signatures.
The agent reads signatures over the zone’s network, so each Edera pod’s egress policy must allow:
- the registry of each image that the agent checks, unless the daemon reads that registry for it (see Private registries);
- the Sigstore TUF repository,
https://tuf-repo-cdn.sigstore.dev, unless you settrusted-root-path.
Find the identity that signs your images
Check that cosign verifies the image with the identity that you will allow. For an image signed in GitHub Actions:
cosign verify \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
--certificate-identity-regexp '^https://github.com/example/app/' \
ghcr.io/example/app:v1Attestations are required by default. Check them too, or set ignore-attestations in the next step:
cosign verify-attestation --type spdxjson \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
--certificate-identity-regexp '^https://github.com/example/app/' \
ghcr.io/example/app:v1The attestor reads cosign v3 bundles attached as OCI referrers, and the older .sig and .att tags.
Configure the daemon
On each Edera node, add a [spire.sigstore] table to /var/lib/edera/protect/daemon.toml. The table turns the checks on:
[spire.sigstore]
allowed-identities = { "https://token.actions.githubusercontent.com" = ["^https://github\\.com/example/"] }
# skipped-images = ["registry.k8s.io/pause@sha256:..."]
# ignore-attestations = true
# trusted-root-path = "/var/lib/edera/protect/spire/trust/sigstore-trusted-root.json"
# [spire.sigstore.registry-credentials."registry.example.org"]
# username = "reader"
# password-path = "/etc/edera/registry-password"| Key | Default | Meaning |
|---|---|---|
allowed-identities | empty | OIDC issuer to the subjects that may sign. A value that contains one of *+?^${}[]|() is a regular expression. Each issuer must list at least one subject. If the table is empty, any keyless signature counts, so set it. |
skipped-images | empty | Image refs, as the pod status shows them in imageID, that are not checked. A skipped image gets no signature selectors. |
ignore-attestations | false | Do not require an attestation, and do not report image-attestations:verified. |
ignore-sct | false | Do not require a Signed Certificate Timestamp in the certificate. |
ignore-tlog | false | Do not require a transparency log entry. The certificate must then be valid at the time of the check, and the log selectors are not reported. |
trusted-root-path | unset | A Sigstore trusted_root.json on the host, for a private Sigstore deployment. Without it, each agent fetches the public-good root over TUF. |
registry-credentials | unset | Registry host to a username and a password-path. See Private registries. |
The attestor never queries the transparency log. A signature must carry its own log entry unless ignore-tlog is set, and cosign adds one when it signs.
Restart the daemon:
sudo systemctl restart protect-daemonThe daemon refuses a table with an unknown key, an issuer with no subjects, or a value that contains ${ or a control character. The settings apply to each SPIRE agent that the daemon launches after the restart. An agent that is already running keeps its old configuration, so delete and recreate its pod.
Private registries
For each host under registry-credentials, the daemon reads signatures for the zone agents with those credentials, so the credentials stay on the node and never enter a zone. password-path names a file that holds only the password. If group or others can read it, the daemon does not start, so set its mode to 0600.
The daemon reads only the repositories of images that it started in the asking zone, and only by digest, by the tags those images name, or by the sha256-<digest> tags that cosign uses. The agents read every other registry themselves, without credentials.
Name each registry as image references name it, for example ghcr.io. docker.io, index.docker.io, and registry-1.docker.io are one host. Credentials for docker.io require the default [oci] docker-hub-mirror, because the daemon would otherwise send them to the mirror.
Each signature or attestation layer can be at most 128 MiB, the cosign default. All reads for one image must finish within the 10-second deadline of the check.
Write identities that require a signature
Add signature selectors to a ClusterSPIFFEID. This one gives the api container of app: payments pods its identity only when its image is signed and attested by the release workflow of example/payments:
apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterSPIFFEID
metadata:
name: payments-api-signed
spec:
className: spire
podSelector:
matchLabels:
app: payments
spiffeIDTemplate: "spiffe://{{ .TrustDomain }}/payments/api"
workloadSelectorTemplates:
- "k8s:container-name:api"
- "k8s:image-signature:verified"
- "k8s:image-attestations:verified"
- "k8s:image-signature-issuer:https://token.actions.githubusercontent.com"
- "k8s:image-signature-subject:https://github.com/example/payments/.github/workflows/release.yml@refs/tags/v1.4.0"A selector must equal the certificate value exactly, and cannot hold a regular expression. To accept a set of subjects, list them in allowed-identities, and select on image-signature:verified and the issuer.
To give one identity to a zone only while every workload in it runs an image from the same signer, use the zone-wide selectors:
workloadSelectorTemplates:
- "edera:zone-image-signature:verified"
- "edera:zone-image-signature-subject:https://github.com/example/payments/.github/workflows/release.yml@refs/tags/v1.4.0"Signature selectors
Both the k8s and edera workload attestors report the per-image selectors, so each is available with either prefix:
| Selector | Value |
|---|---|
image-signature:verified | A signature from an allowed identity verified |
image-attestations:verified | An attestation from an allowed identity verified |
image-signature-subject:<subject> | The signing certificate’s subject, once for each verified signature |
image-signature-issuer:<issuer> | The signing certificate’s OIDC issuer, once for each verified signature |
image-signature-value:<base64> | The signature |
image-signature-log-id:<hex> | The log entry’s log ID |
image-signature-log-index:<n> | The log entry’s index |
image-signature-integrated-time:<unix time> | The log entry’s integrated time, for a Rekor v1 entry |
image-signature-signed-entry-timestamp:<base64> | The log entry’s signed entry timestamp |
The zone-wide selectors are Edera’s, and only the edera attestor reports them. They cover every running workload in the zone except the SPIRE agent:
| Selector | Value |
|---|---|
edera:zone-image-signature:verified | Every running workload’s image has a verified signature from an allowed identity |
edera:zone-image-attestations:verified | Every running workload’s image has a verified attestation |
edera:zone-image-signature-subject:<subject> | A subject that signed every running workload’s image |
edera:zone-image-signature-issuer:<issuer> | An issuer whose certificates signed every running workload’s image |
The subject and issuer selectors are reported separately, so two different signatures can supply them. Keep allowed-identities to identities that you trust on their own.
How a failed check behaves
- A container whose image is not signed by an allowed identity, has no attestation when one is required, or whose signatures cannot be read, gets no
k8s:oredera:selectors. Every entry that uses them stops matching, including the chart’s default identity. Upstream SPIRE behaves the same way. - Another workload in the zone that does not verify removes only the zone-wide selectors. Each container’s own image is still judged on its own.
- A verified image is remembered until the SPIRE agent restarts. A failure is not remembered, so the next request checks again.
- Each check has a 10-second deadline. A registry that does not answer fails the check.
Troubleshoot signature checks
Signature failures appear in the SPIRE agent’s log in the pod’s zone. On the node, the agent is the workload spire-agent-<zone uuid>:
protect-ctl zone list
protect-ctl workload attach spire-agent-<zone uuid>| Message in the agent log | Cause |
|---|---|
no signature from an allowed identity verified for image "<image>" | No signature, a signer that allowed-identities does not allow, or a signature that did not verify. The message lists each reason. |
no attestation from an allowed identity verified for image "<image>" | The image is signed but has no attestation. Add one, or set ignore-attestations = true. |
image "<image>" names no repository to find its signatures in | The container runtime reported no image or image ref. |
failed to load the sigstore trusted root | The agent could not fetch the root from the Sigstore TUF repository. At agent start, trusted-root-path is not a valid trusted root. |
the sigstore trusted root is still loading | The TUF repository did not answer within the check’s deadline. The fetch continues, and a later check uses it. |
A failed read through the daemon appears in the daemon’s log on the node:
journalctl -u protect-daemon | grep "SPIRE registry read"Run cosign tree <image> to see what an image carries. Under CRI-O, an image that CRI-O knows only by its ID, such as one built or loaded on the node, has no image ref, and is refused unless it is skipped.
Operations
- Node key renewal. Automatic. Each node key is an agent SVID with the server’s agent SVID lifetime, one hour by default. The enroller renews it at half that. That lifetime is also how long a stolen node key stays usable, so keep the chart’s agent SVID TTL at one hour or less.
- Removing a node. Delete its Node object. The server then refuses its evidence at once, because the node key names the node’s UID. An enroller cannot renew for a node that no longer exists.
- SPIRE CA rotation. Nothing to do. The chart rotates the CA daily.
trust-bundle-pathis only the first trust anchor. The daemon and the enroller each read the server’s current bundle over TLS that the bundle they already hold authenticates. The daemon reads it every 5 minutes and at each zone launch, and the enroller every 5 minutes and at each enrollment. SPIRE publishes each new CA hours before it signs with it. If a node is off for longer than the CA lifetime, one day by default, delete its enroller pod. The new pod’s init container copies the current bundle from the ConfigMap. - Evidence lifetime. The server refuses evidence valid for longer than the plugin’s
max_evidence_lifetime_seconds(default 86400). Lower it inplugin_datato enforce a shorter limit than the daemons use. - Server outside the cluster. The Edera plugin must read pods and nodes from the Kubernetes API. Run a nested SPIRE server in the cluster with the plugin, and make the outside server its upstream authority.
- Zones that are not a pod’s. A node key from SPIRE can sign only for a pod, because only the pod gives the node’s name. Zones started outside Kubernetes need a node certificate from your own CA, the plugin’s
node_identity = "certificate"mode.
Security notes
- Who controls node identity. SPIRE’s CA issues the node keys. Anyone who can run a pod, or mint a token, as the enroller’s service account can enroll the node that pod runs on. Treat the rights in
$NAMESPACEas node credentials, as upstream SPIRE already requires for its own agent. - The bundle ConfigMap is a trust anchor. Whoever can change
spire-bundlein$NAMESPACEdecides what each node trusts at its enroller’s next pod start, as for the standard SPIRE agent. The same namespace rights can already enroll nodes. For an out-of-band check, leave out the init container, and installserver-bundle.pemon each node yourself. - A zone cannot enroll. The plugin refuses an enroller token from any pod that sets a runtime class. A workload in a zone never gets a node key, even if it runs with the enroller’s service account, and whatever the Edera RuntimeClass is named.
- A node key signs for its own node only. It can claim any Edera pod on its node, and no pod on any other node or in any other cluster. It grants nothing else: the enroller’s agent has no selectors and no entries.
spiffe://<trust domain>/edera/is reserved. It holds each pod zone’s delegate identity, which can fetch every identity in that zone. Only theClusterSPIFFEIDfrom step 3 may render IDs under it.- Parents. Keep entries for Edera pods under the pod’s agent, which the registrar does. If you write a node alias by hand, select on
edera-hypervisor:node-idtogether withedera-hypervisor:zone-idoredera-hypervisor:k8s-pod-uid. An alias that spans many zones gives its identities to every zone under it.
Troubleshooting
Read the SPIRE server log first:
kubectl -n "$NAMESPACE" logs spire-server-0 -c spire-server| Symptom | Cause |
|---|---|
node enrollment refused: ... is not an allowed enroller service account | The enroller’s service account is not in service_account_allow_list. |
node enrollment refused: ... must run on the default runtime | The enroller pod sets runtimeClassName. Remove it. |
node enrollment refused: the token is not for audience "edera-node" | The chart’s token.audience differs from the plugin’s audience. |
node enrollment refused: the token was issued on node ... | The node was replaced under the same name, and a token from the old node was used. Delete the old enroller pod. |
The enroller logs the server's certificate is not for spiffe://.../spire/server | The enroller’s trust domain differs from the chart’s trustDomain. |
The enroller pod stays in Init | The server has not published its bundle to spire-bundle, or the ConfigMap has another name or format. See chart values. |
evidence did not verify: node SVID ... is not an enrolled node | The node key is not from the enroller. Check node-key-path and node-certificate-path. |
statement is for trust domain "a", not "b" | The daemon’s trust-domain differs from the chart’s trustDomain. |
statement is valid for ..., longer than ... | evidence-lifetime-seconds is above the plugin’s max_evidence_lifetime_seconds. |
pod not confirmed: ... could not read node | The server’s service account cannot read nodes. Keep the chart’s PSAT node attestor enabled. |
The daemon logs waiting for the SPIRE node key | The node has not enrolled. Check the enroller on that node. |
| The agent is listed, but the pod has no entry under it | The pod’s node selector lacks runtime: edera, so its entry went under a node agent. Check the RuntimeClass (step 1). |
| Pods lost their default identity | The step 3 object is missing its annotation, or the registrar is not the Edera image. |
| No mounted identity, but the Workload API works | The step 3 object is missing, or its className differs from the registrar’s. |
| The daemon does not start | Check journalctl -u protect-daemon. server-address without node-key-path, node-certificate-path, and trust-bundle-path stops it. |
The daemon logs starting without a SPIRE node key | The node key or its certificate is missing, cannot be read, is readable by others, or does not match. The daemon runs, and uses the pair once a good one appears. |
| Entries written by hand disappear | addEntryIDPrefix is off, so the registrar deletes every entry that no ClusterSPIFFEID declares. Keep the chart default, which turns it on. |
See also
- Istio mTLS with SPIRE identities: use these identities for Istio sidecar mTLS and policy.
- Workload identity with SPIFFE and SPIRE: the SPIRE architecture, selectors, and the
protect-ctlflow. daemon.tomlreference:[spire]: every[spire]option and default.- SPIRE Helm charts: the upstream
spireandspire-crdscharts. - Kubernetes workload attestor
sigstoresupport: the upstream behavior that Require signed images follows.