Service mesh for military zero-trust: mTLS, policy enforcement, and observability in defense clusters

A Kubernetes cluster running defense microservices is not inherently zero-trust. The default pod networking model allows any pod to initiate a TCP connection to any other pod on the same cluster network, which means a single compromised container can reach authentication services, data stores, and command interfaces that it has no legitimate reason to access. A service mesh addresses this not by reconfiguring network ACLs — which are fragile in dynamic container environments — but by inserting cryptographic identity and policy enforcement directly into the data path of every service-to-service call. This article examines what that means in practice for defense clusters: how SPIFFE/SPIRE integrates with zero-trust architecture for military networks, how intent-based policy replaces perimeter thinking inside the cluster, and how to maintain full observability without any telemetry crossing the classification boundary.

Why perimeter security fails inside defense microservice clusters

The perimeter security model assumes that traffic inside a trusted network boundary is safe. In a monolithic application, that assumption has some validity: there is one process, one trust boundary, and relatively few internal communication paths to secure. In a microservice architecture deployed on Kubernetes, the assumption collapses entirely. A cluster running 20 services has on the order of 400 possible service pairs. Not all of them should communicate, but without explicit enforcement at the service level, all of them can. A vulnerability in one service — a dependency with a remote code execution flaw, a misconfigured environment variable exposing credentials, an unpatched base image — hands an adversary lateral movement capability across the entire cluster network.

The failure mode is not theoretical. Supply-chain compromises that target container workloads have demonstrated that once an adversary has code execution inside a pod, they probe the cluster network aggressively. Without mutual authentication, there is nothing preventing the compromised pod from calling internal APIs directly: it already has a valid cluster IP, its requests arrive from a trusted CIDR range, and the target service has no way to verify the caller's identity. Traditional network segmentation via Kubernetes NetworkPolicy helps at the IP/port layer but does nothing to prevent a legitimate pod's identity from being impersonated within the permitted address range, nor does it enforce any application-layer constraints on what operations are permitted.

A service mesh solves this by making cryptographic workload identity mandatory for every connection. Each sidecar proxy presents a SPIFFE Verifiable Identity Document (SVID) — a short-lived X.509 certificate encoding the pod's service account identity — and requires the remote sidecar to present a valid SVID in return before any application data flows. The identity check happens at the transport layer, transparent to the application, and cannot be bypassed by an attacker who controls only the application process inside the container. Lateral movement is reduced from "reach any pod on the cluster network" to "reach only pods whose policy explicitly permits your SVID as a source."

Choosing a service mesh: Istio vs Linkerd vs Cilium in air-gapped environments

Three service mesh implementations are production-viable for defense Kubernetes deployments, each with a different trade-off profile. Istio is the most feature-complete: its Envoy-based data plane supports rich L7 traffic management including fault injection, retries, circuit breaking, request mirroring, and fine-grained AuthorizationPolicy. The control plane — istiod — integrates natively with SPIFFE/SPIRE for external certificate issuance, and its telemetry pipeline emits OpenTelemetry-compatible metrics and traces from every sidecar. The cost is operational complexity: Istio has a steeper learning curve, a larger control plane footprint, and Envoy sidecars that consume more memory per pod than alternatives. For clusters with 16+ GB of RAM per node and teams experienced with Envoy configuration, Istio's policy expressiveness is hard to match.

Linkerd takes the opposite approach. Its data plane proxy is written in Rust and designed to be ultralight: each linkerd-proxy sidecar uses 10–20 MB of resident memory and adds under 1 ms of latency to each hop, compared to 50–100 MB and 0.5–3 ms for Envoy. Linkerd's zero-configuration mTLS is on by default for all meshed traffic without any PeerAuthentication manifest required, which reduces the risk of a misconfigured plaintext exception. The trade-off is that Linkerd's policy model is less expressive at the HTTP method and path level, and its control plane integrates with external SPIRE issuers less directly than Istio's. For resource-constrained edge clusters with limited per-node RAM, Linkerd's proxy efficiency is a significant operational advantage.

Cilium operates at the eBPF layer rather than as a sidecar. By embedding policy enforcement into the Linux kernel's eBPF programs attached to each pod's network interface, Cilium achieves near-zero per-workload overhead for network policy enforcement and basic mTLS. Its CiliumNetworkPolicy resource supports SPIFFE-identity-aware rules through integration with the SPIRE API, and the Hubble observability component provides L4/L7 flow visibility within the cluster. Cilium's strength is performance and its seamless replacement of kube-proxy; its limitation is that L7 HTTP/gRPC policy enforcement via eBPF is less mature than Istio's Envoy-based equivalent for complex authorization scenarios. In practice, many defense clusters run Cilium as the CNI for network policy and encryption, with Istio or Linkerd layered on top for L7 observability and fine-grained service-level AuthorizationPolicy.

Mutual TLS: certificate issuance, rotation, and SPIFFE/SPIRE integration

Mutual TLS in a service mesh means both sides of every connection authenticate each other with X.509 certificates before any application data is exchanged. The critical question for a defense deployment is how those certificates are issued, how frequently they rotate, and who controls the root of trust. Native Istio uses its own built-in certificate authority (the istiod CA), which issues workload certificates valid for 24 hours by default. For a defense deployment, this is inadequate on two counts: the root key lives inside the cluster (a compromised istiod process exposes all workload identities), and 24-hour TTLs are too long for environments where rapid credential revocation may be required.

SPIRE provides the correct architecture for defense contexts. The SPIRE server runs as a StatefulSet with its trust bundle and registration entries persisted to a datastore that can be kept outside the cluster. The SPIRE agent runs as a DaemonSet on every node and handles workload attestation by verifying each pod's Kubernetes service account token against the API server before issuing an SVID. Issued SVIDs have configurable TTLs — production defense deployments use 1-hour TTLs for standard workloads and 15-minute TTLs for workloads handling classified data — and the SPIRE agent handles rotation automatically, pushing a new SVID to the workload socket before the current one expires. The mesh sidecar reads the SVID via the SPIFFE Workload API socket and presents it on all outbound TLS connections without any application code changes.

Integrating SPIRE with Istio requires configuring istiod to use SPIRE as its certificate provider via the upstream CA plugin interface. Istio's CustomCA configuration points istiod at the SPIRE server's GRPC port, delegating all SVID issuance and rotation to SPIRE while Istio handles the PeerAuthentication and AuthorizationPolicy enforcement. This separation of concerns is important: SPIRE owns the identity lifecycle (attestation, issuance, rotation), while Istio owns the policy lifecycle (which identities may communicate on which paths). An operator revoking a workload's access removes its SPIRE registration entry, which causes the next SVID rotation to fail, cutting off the workload from the mesh at the next certificate renewal without requiring any policy manifest changes.

Intent-based traffic policy: allow-listing east-west communication

The default-deny principle requires that no service-to-service traffic is permitted unless explicitly allowed by policy. In Istio, this is implemented with a cluster-wide AuthorizationPolicy manifest that has an empty rules block and action: DENY — which blocks all traffic — followed by narrow allow policies for each legitimate service pair. Each allow policy names the source principal (the SPIFFE URI of the calling service's service account), the destination service, and the precise set of HTTP methods or gRPC service names that the call is authorized for. A policy authorizing a sensor data ingestion service to POST to the data fusion API, for example, does not also permit it to GET historical records or call the administration endpoint, even though both run on the same destination service port. The granularity is at the operation level, not merely the TCP connection level.

Writing and maintaining this policy set requires discipline. A cluster that grows from 10 to 30 services over 18 months accumulates hundreds of allow entries, and stale entries for decommissioned services become latent attack surface. Two practices prevent this accumulation. First, treat AuthorizationPolicy manifests as code: store them in the same version-controlled repository as the application manifests, require peer review for every new allow entry, and audit the policy set quarterly against the list of currently running services. Second, use the mesh's access logging — every denied request generates an access log entry in the sidecar that includes the source SVID, destination, and operation — to identify legitimate traffic paths that are missing a policy rather than silently bypassing the mesh. This logging also provides the forensic trail required by many defense security frameworks to demonstrate that access control decisions are auditable.

Observability without data exfiltration: in-cluster metrics and tracing

A service mesh generates rich telemetry automatically from its sidecar proxies: per-service request rates, error rates broken down by HTTP status code and gRPC status, latency percentiles (p50, p95, p99), active connection counts, and TLS handshake metrics. In a commercial cloud deployment, this telemetry is typically forwarded to a SaaS observability platform. In a classified military cluster, any telemetry pipeline that sends data outside the classification boundary is a potential exfiltration vector and is architecturally prohibited. The entire observability stack must be deployed inside the cluster boundary and must itself be access-controlled by the same mesh policies that govern application traffic.

The standard in-cluster observability stack for a service mesh consists of Prometheus for metrics collection, Grafana for visualization, and Jaeger or Tempo for distributed tracing. Prometheus scrapes the Envoy stats endpoint (port 15090) or the linkerd-proxy metrics endpoint on every meshed pod at a configurable interval. Grafana connects exclusively to the in-cluster Prometheus and Loki instances — no external data source connections are configured. Jaeger receives trace spans forwarded by the Envoy sidecar over the Zipkin or OpenTelemetry protocol to a cluster-local collector endpoint. All persistent storage for metrics, traces, and logs uses cluster-local PersistentVolumes backed by encrypted block storage. The monitoring namespace itself carries a NetworkPolicy that blocks all egress to external IP ranges, making it architecturally impossible for telemetry to leave the cluster even if a misconfigured pipeline were introduced.

For Kubernetes clusters hardened for defense workloads, it is worth noting that the observability stack must also be sourced from air-gapped registries. Prometheus, Grafana, Jaeger, and Loki images must be mirrored into the internal registry with verified digests before the cluster is placed behind the classification boundary. Image pull policies must be set to Never or IfNotPresent with the understanding that the images will never be refreshed from an external source. Any update to the observability stack requires a formal change management process that brings updated images through the air-gap procedure rather than a simple Helm chart upgrade against a public chart repository.

Sidecar overhead in resource-constrained defense edge clusters

Edge clusters deployed close to operations — running on small form-factor servers or ruggedized compute platforms with 32–64 GB of RAM and 4–8 nodes — face a resource budget that makes sidecar overhead a genuine engineering constraint rather than a theoretical concern. An Istio Envoy sidecar at steady state consumes 50–100 MB of resident set size and up to 0.5 vCPU during high-throughput periods. A cluster running 40 pods consumes 2–4 GB of aggregate sidecar memory, roughly 6–12% of a 32 GB cluster's total RAM. The control plane itself — istiod, Prometheus, Grafana, Jaeger — adds another 2–4 GB, bringing total mesh infrastructure overhead to 15–25% of cluster memory before any application workloads are considered.

Three options reduce this overhead in constrained deployments. First, Linkerd's Rust-based proxy is the most efficient sidecar available: 10–20 MB per pod rather than 50–100 MB for Envoy, and latency overhead under 1 ms per hop. For clusters where L7 policy granularity requirements can be met by Linkerd's policy model, the memory savings are substantial. Second, Istio's ambient mesh mode (available from Istio 1.22 onward) eliminates per-pod sidecars entirely, replacing them with a per-node ztunnel proxy that handles mTLS for all pods on the node. Ambient mode reduces per-pod overhead to near zero at the cost of moving policy enforcement from per-workload sidecars to per-node proxies, which provides slightly coarser policy granularity for workloads that need per-request authorization decisions. Third, selective injection — injecting sidecars only into pods that require L7 policy enforcement, and using Cilium's eBPF-based L4 encryption for the rest — balances enforcement granularity against resource consumption for heterogeneous cluster workloads.

The interaction between sidecar resource consumption and secrets management in defense CI/CD pipelines is worth noting explicitly. Secret injection sidecars (Vault Agent Injector, Secrets Store CSI Driver) add another container per pod. An edge cluster that runs both a mesh sidecar and a secrets injection sidecar per application pod has two infrastructure containers for every application container, doubling the container count and adding 60–120 MB of infrastructure overhead per pod pair. Consolidation is possible: the SPIRE agent can serve as both the mesh identity issuer and the workload secret delivery mechanism for secrets that can be expressed as SVID-authorized Vault roles, eliminating the separate secrets injection sidecar for those workloads.

Integrating service mesh policy with RBAC and secrets management

Kubernetes RBAC governs who can read and modify Kubernetes API objects — including the AuthorizationPolicy, PeerAuthentication, and mesh configuration resources that define the security posture of the service mesh. If the RBAC configuration permits a broad set of principals to modify AuthorizationPolicy objects, the mesh's access control layer can be weakened by any of those principals. A correct defense deployment treats mesh policy manifests as high-value security objects: only the CI/CD service account that applies manifests from the version-controlled policy repository should have write access to AuthorizationPolicy and PeerAuthentication resources, and only cluster administrators should be able to modify SPIRE registration entries. All other principals — including individual application service accounts — should have read-only or no access to these objects.

The relationship between the mesh identity layer and the secrets management layer is bidirectional. The mesh uses SPIFFE SVIDs to authenticate service-to-service calls at the transport layer. Secrets management (Vault, Kubernetes Secrets Store CSI Driver) uses those same SVID-based identities to determine which workloads may receive which secrets at the application layer. A workload with SPIFFE ID spiffe://cluster.local/ns/fusion/sa/data-processor can be mapped to a Vault role that grants read access to exactly the database credentials that service needs — and nothing else. This means the mesh identity system and the secrets delivery system share a single authoritative source of workload identity, eliminating the risk of a mismatch where a workload is permitted network access but not the secrets required to use it, or vice versa. Maintaining this alignment requires that SPIRE registration entries, Vault role definitions, and mesh AuthorizationPolicy manifests are all updated together when a service is added, modified, or decommissioned.

Audit logging ties the three systems together for compliance and incident response. The mesh sidecar logs every access decision — allow or deny — with the verified source SVID, destination, operation, and timestamp. Vault logs every secret access with the requesting identity and the secret path accessed. Kubernetes API server audit logs record every RBAC-governed API call. Together, these three log streams provide a complete audit trail from "which network call was made" through "which secrets were accessed as a result" to "who changed the policy that governed both." For defense environments operating under formal accreditation requirements, this traceability across the identity, access control, and secrets layers is often a non-negotiable compliance requirement rather than an operational nicety.