Backup and recovery for classified cloud workloads: RTO, RPO and cryptographic continuity

Classified military cloud workloads are not abstract assets. They run the targeting pipelines, communications fabric, and intelligence fusion layers that determine whether a unit maintains situational awareness under fire. When those workloads fail — and hardware fails, facilities lose power, and adversaries probe every reachable surface — the organization needs a tested, executable recovery plan that restores them within the time the mission can tolerate. Backup and disaster recovery for classified systems is not a scaled-down version of commercial DR; it is a distinct engineering discipline shaped by accreditation constraints, cryptographic key dependencies, and the operational reality that the systems most in need of rapid recovery are the ones hardest to restore under pressure.

This article examines the full recovery stack for classified cloud workloads: how to derive RTO and RPO targets from mission criticality tiers, how to design backup architecture that satisfies classification boundaries, how to manage encryption keys so that they survive a primary site failure, how to back up databases and Kubernetes clusters, how to test recovery in environments that restrict physical access, and how to re-establish cryptographic continuity once systems are restored. The treatment is technical and operational — the decisions here belong to platform engineers, information system security officers (ISSOs), and program architects working together.

RTO and RPO requirements for military C2 and ISR systems

Recovery Time Objective (RTO) and Recovery Point Objective (RPO) are not IT service level agreements imported from a commercial template. For classified defense systems they are derived from operational tempo — the rhythm at which commanders need current data to make decisions — and from mission criticality, which determines how long a capability can be absent before the mission degrades to an unacceptable level.

A practical criticality framework assigns systems to three tiers:

• Tier 1 — Mission-essential C2 and real-time ISR. Command and control platforms, real-time sensor fusion, and active targeting systems. RTO: under four hours. RPO: under 15 minutes. A C2 system that is unavailable for more than four hours during an active operation impairs the commander's ability to issue, track, and revise orders. An RPO greater than 15 minutes means a potential loss of recent targeting or situational data that cannot be reconstructed.
• Tier 2 — ISR analysis and mission-support systems. Intelligence analysis workstations, communications recording, and logistics management. RTO: 8–24 hours. RPO: one to four hours. These systems support mission planning and assessment rather than real-time execution; their absence degrades efficiency but does not immediately halt operations.
• Tier 3 — Administrative and archival systems. Personnel systems, archival storage, and administrative applications. RTO: 48–72 hours. RPO: 24 hours. Extended unavailability is operationally tolerable; data loss of up to one business day is acceptable.

The critical design implication of the tier assignment is that Tier-1 RTO targets — sub-four-hour restoration — are achievable only with hot or warm standby architectures. Cold backup (tape or disk backup with no running standby) introduces recovery steps that, taken together, cannot complete in four hours: media retrieval, infrastructure provisioning, operating system restore, application layer restore, security control verification, and ISSO sign-off. A program that believes it has a four-hour RTO through cold backup alone has not modeled the actual recovery procedure.

The RTO budget must be broken down by phase and summed before adoption as a target:

Recovery phase	Hot standby	Warm standby	Cold backup
Failover decision and authorization	5–15 min	15–30 min	30–60 min
Media retrieval / key recovery	N/A (live replica)	15–30 min	60–180 min
Infrastructure and OS restore	0–15 min	30–60 min	60–120 min
Application and data restore	0–5 min	20–60 min	60–240 min
Security control verification + ISSO sign-off	30–60 min	60–90 min	60–120 min

The security control verification step — confirming that classification markings, audit logging, access controls, and cryptographic bindings are all functioning correctly on the restored system — is frequently omitted from commercial RTO models. For classified systems, it is mandatory before return to operations. An accurate RTO target accounts for it.

Backup architecture for classified workloads

Classified workload backup architecture starts from two non-negotiable constraints: classification boundary isolation and accreditation continuity. Each classification enclave requires physically separate backup infrastructure — separate storage nodes, separate media, separate backup software instances if the software uses a shared management plane. Consolidated backup infrastructure that spans enclaves is a compliance violation regardless of whether backup data is encrypted, because the shared management plane creates a potential covert channel and an expanded attack surface.

Accreditation continuity means that the restore environment — the infrastructure to which classified data is restored during a disaster — must carry a current Authorization to Operate before the disaster, not just after it. The most common classified DR failure in post-incident reviews is not a missing backup; it is a backup that exists but cannot legally be restored within the required timeframe because the restore environment's ATO has lapsed.

Immutable backup storage is a mandatory control for classified workloads at Tier 1 and Tier 2. Immutability — enforced at the hardware or firmware level through write-once media or object lock in compliance mode — ensures that a ransomware actor or malicious insider who compromises backup infrastructure cannot delete or alter backup sets. Software-enforced WORM that can be overridden by a privileged account does not satisfy this requirement. For on-premises classified storage, hardware WORM tape (LTO with WORM cartridges) or a disk appliance with firmware-level immutability is the appropriate choice. For sovereign classified cloud deployments, object storage with S3-compatible object lock in compliance mode provides equivalent protection.

A three-layer architecture satisfies the full range of recovery scenarios:

• Layer 1 — Local immutable backup. Continuous or hourly incremental backups to local WORM storage within the accredited facility. Protects against operational errors: accidental deletion, database corruption, ransomware. Fastest restore path for non-catastrophic failures.
• Layer 2 — Synchronous replication to warm standby. For Tier-1 systems, database transaction logs and critical state are replicated to a secondary node in the same or a co-located accredited facility. This layer supports sub-four-hour RTO. Replication is within the accreditation boundary — the secondary node is part of the same accredited environment.
• Layer 3 — Periodic offsite copy to DR facility. Weekly or monthly encrypted backup copies transferred to a physically separate accredited facility. This layer protects against catastrophic primary site loss. For tactical edge cloud disconnected ops, this transfer is physical — encrypted media transported by authorized courier — and the courier transit time must be included in the RTO calculation for the DR scenario it covers.

Air-gapped DR sites introduce a specific design challenge: the offsite copy is always behind the primary by the interval between physical transfers. A program that transfers backup media to its DR site weekly has a potential data loss window of up to seven days for the scenario where the primary site is destroyed. This gap must be documented, accepted by the mission authority, and reflected in the system's contingency plan — not hidden in the architecture.

Encrypting backup data: key management through recovery

Every backup set for a classified workload must be encrypted at rest using AES-256 (or the national equivalent approved for the system's classification level). The harder problem is not the encryption itself — it is ensuring that the decryption keys survive a primary site failure and can be accessed at the DR site within the recovery time budget.

The recommended key hierarchy for classified backup encryption has three levels:

• Key Encryption Key (KEK). A master key held in a Hardware Security Module (HSM) within the accredited facility. The KEK never leaves the HSM in plaintext. Access to the KEK requires multi-party authorization — at minimum two authorized personnel with separate HSM authentication credentials (an m-of-n quorum scheme, typically 2-of-3 or 3-of-5).
• Data Encryption Key (DEK). A unique AES-256 key generated per backup job. The DEK encrypts the backup data. After the backup job completes, the DEK is encrypted (wrapped) by the KEK inside the HSM and the wrapped DEK is stored alongside the backup metadata. The plaintext DEK is never written to disk.
• Key escrow at the DR site. The KEK is synchronized to a secondary HSM at the DR site, either through continuous HSM cluster replication or through a periodic key backup procedure. The secondary HSM holds the KEK in an equal-security environment and releases it to authorized restore operators during a declared disaster, allowing on-site DEK unwrapping and backup decryption.

The escrow synchronization frequency determines the maximum staleness of the DR-site KEK. For rotating KEKs (annual or more frequent rotation), the escrow update must occur within one rotation period. The escrow procedure — including the authentication and authorization steps required for the DR-site HSM to accept the updated key — must be documented, and the documentation must be stored at the DR site (not only at the primary site).

For deeper context on HSM selection and post-quantum key management HSM architectures that provide long-term resistance to quantum-enabled attacks on stored ciphertext, refer to the linked treatment. The key hierarchy above is compatible with post-quantum KEK algorithms (CRYSTALS-Kyber or ML-KEM at CNSA 2.0 levels) without changing the structural relationship between the layers.

A DR runbook that has never been exercised end-to-end — including the DR-site HSM authentication and DEK unwrapping — has not validated its most failure-prone step. Key recovery must be exercised as a named step in every full restore rehearsal, not left as an assumed capability.

Database backup strategies for operational C2 systems

Operational C2 systems typically persist state in relational databases: PostgreSQL is the dominant open-source choice for accredited defense cloud deployments. The standard commercial backup of "daily full dump plus nightly differentials" does not meet the RPO requirements of Tier-1 systems — an RPO of 15 minutes requires a continuous backup mechanism that captures every committed transaction.

PostgreSQL's Write-Ahead Log (WAL) provides that mechanism. Every committed database change is written to a WAL segment before it is applied to the data files. By continuously archiving WAL segments to accredited backup storage immediately after they are written, you accumulate a complete change log that can be replayed forward from any base backup to any point in time — right up to the last archived segment before a failure. This is Point-in-Time Recovery (PITR).

Configuration in postgresql.conf for continuous WAL archiving with encryption:

wal_level = replica
archive_mode = on
archive_command = '/opt/backup/encrypt-wal.sh %p %f'
archive_timeout = 60        # force segment switch every 60 seconds maximum
restore_command = '/opt/backup/decrypt-wal.sh %f %p'
recovery_target_time = '2026-06-25 14:30:00 UTC'  # set in recovery.conf

The encrypt-wal.sh script should encrypt the WAL segment using the HSM-backed DEK before writing to the archive location. The archive_timeout of 60 seconds ensures that even during low-write periods, WAL segments are archived at least every minute, bounding the RPO at approximately one minute under normal conditions.

For C2 systems composed of multiple microservices sharing a distributed state — a common pattern where targeting data flows between a sensor fusion service, a decision support service, and a communications gateway — backup consistency requires that snapshots of all service databases are taken at the same logical point in time. A backup set where the targeting update exists in the fire control database but not yet in the ISR fusion database produces a logically inconsistent restore state. Consistent snapshots across microservices are achieved through:

• A distributed snapshot coordinator that issues a quiesce signal to all services, waits for in-flight transactions to drain, triggers snapshots on all databases simultaneously, then releases the quiesce.
• Pre-backup hooks in the container orchestrator that call each service's quiesce API before the volume snapshot is triggered.
• A sequence number or global transaction ID stamped into every snapshot set, allowing restore procedures to verify that all components of a restore set share the same logical timestamp before committing to restore.

Kubernetes workload backup

Velero is the standard open-source tool for Kubernetes workload backup in both commercial and defense contexts. In a classified air-gapped cluster, Velero's deployment requires specific adaptations: all Velero container images, plugin images (particularly the CSI plugin and any object storage provider plugin), and the Velero CLI binary must be pre-staged in the cluster's local image registry before a disaster, because the cluster cannot pull from external registries during recovery.

Velero backs up Kubernetes API objects — Deployments, DaemonSets, Services, ConfigMaps, Secrets, PersistentVolumeClaims, NetworkPolicies, RBAC objects, and custom resources — and triggers CSI volume snapshots for persistent data. A Velero backup schedule for a classified cluster:

apiVersion: velero.io/v1
kind: Schedule
metadata:
  name: classified-cluster-hourly
  namespace: velero
spec:
  schedule: "0 * * * *"          # every hour
  template:
    storageLocation: classified-backup-location
    volumeSnapshotLocations:
      - classified-csi-snapshots
    includedNamespaces:
      - c2-platform
      - isr-fusion
      - comms-gateway
    hooks:
      resources:
        - name: db-quiesce
          includedNamespaces:
            - c2-platform
          labelSelector:
            matchLabels:
              app: postgres
          pre:
            - exec:
                command:
                  - /bin/sh
                  - -c
                  - psql -c "SELECT pg_start_backup('velero', true);"
                timeout: 60s
          post:
            - exec:
                command:
                  - /bin/sh
                  - -c
                  - psql -c "SELECT pg_stop_backup();"
                timeout: 60s
    ttl: 720h                    # 30-day retention

What Velero does not back up: etcd state (Velero reads from the API server, not etcd directly), node-level OS configuration, control-plane binaries, and data written to node-local storage outside of persistent volumes. etcd must be backed up separately. For a three-node control plane, run the following on each control-plane node and encrypt the output:

ETCDCTL_API=3 etcdctl snapshot save \
  /tmp/etcd-snapshot-$(date +%Y%m%d-%H%M%S).db \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key

# Encrypt snapshot before archiving
gpg --symmetric --cipher-algo AES256 \
  --batch --passphrase-file /etc/backup/etcd-key.txt \
  /tmp/etcd-snapshot-*.db

# Verify snapshot integrity
etcdctl snapshot status /tmp/etcd-snapshot-*.db

etcd snapshots should be scheduled hourly on all control-plane nodes, with the encrypted snapshots written to the same accredited backup storage used for Velero backups. A complete Kubernetes DR strategy requires both Velero (for workload-layer state) and etcd snapshots (for cluster-layer state). Restoring only one produces an unrecoverable cluster — the API objects in etcd and the persistent data in volumes must be consistent with each other.

PersistentVolumeClaim snapshot strategies depend on the storage class used. For CSI-backed storage in classified environments, the storage driver must implement the CSI snapshot interface and the snapshots must be stored in accredited storage. For NFS or legacy storage that does not support CSI snapshots, Velero's file-system backup mode (Kopia-based) can back up PVC data by directly copying files from mounted volumes — slower than CSI snapshots but applicable to any storage type.

Recovery testing in classified environments

Recovery testing in classified environments is more constrained than in commercial settings: you cannot spin up an arbitrary public cloud environment as a restore target, you cannot test with production data outside the accreditation boundary, and you cannot run restore rehearsals during operational hours without advance authorization and a tested rollback plan.

The DR drill schedule for a classified program should follow a three-tier cadence:

• Tabletop exercise — quarterly. The restore team works through the runbook verbally, identifying steps that are unclear, roles that are unstaffed, or dependencies that are undocumented. No systems are touched. Output: an updated runbook and a list of gaps to remediate.
• Functional drill — semi-annual. A subset of the restore procedure is executed in a live environment: for example, restoring a single database from backup and verifying data integrity, or restoring a Velero backup of one namespace and confirming that the application comes up. Partial coverage at lower cost and risk than a full rehearsal.
• Full restore rehearsal — annual minimum, semi-annual for Tier 1. A complete end-to-end restore from backup to a quarantined restore environment. All restore phases are executed, including key recovery, security control verification, and ISSO sign-off. Actual RTO and RPO are measured and compared against targets.

Data integrity verification after restore requires more than confirming that the database starts. For relational databases, integrity verification includes:

# PostgreSQL post-restore integrity checks
# 1. Verify row counts match expected values from pre-backup audit log
psql -c "SELECT schemaname, tablename, n_live_tup
         FROM pg_stat_user_tables ORDER BY schemaname, tablename;"

# 2. Check for constraint violations after restore
psql -c "SELECT conname, conrelid::regclass
         FROM pg_constraint WHERE NOT convalidated;"

# 3. Verify WAL replay reached the target recovery time
psql -c "SELECT pg_last_xact_replay_timestamp();"

# 4. Run application-layer health check
curl -sf https://c2-platform.internal/health/deep | jq '.checks'

RTO measurement methodology must be precise: the clock starts when a disaster is formally declared (not when the failure is first detected — incident detection and declaration may take 15–30 minutes and must be subtracted from the remaining budget). The clock stops when the ISSO formally signs off on the restored system's readiness for classified operation — not when the application returns its first successful health check. The difference between these two interpretations can be 60–90 minutes, which may determine whether a program is meeting its contractual or regulatory RTO commitment.

Cryptographic continuity after recovery

A system that has been restored from backup is not cryptographically identical to the system that was backed up. Depending on when the backup was taken relative to the last key rotation, certificate issuance, or session establishment, the restored system may be operating with stale cryptographic material that is expired, revoked, or inconsistent with the current state of connected systems. Cryptographic continuity is the set of procedures that bring the restored system's cryptographic state into alignment with the operational environment after recovery.

HSM failover re-keying. When the primary HSM fails and the DR-site secondary HSM takes over, the first step is to verify that the secondary HSM's key inventory is current. For HSMs that use continuous cluster replication, the secondary should be current as of the last replication heartbeat — typically within seconds. For HSMs that use periodic key backup, the secondary may be behind by the backup interval. Any keys created or rotated since the last backup are not present in the secondary and must be re-derived or re-issued before the systems that depend on them can operate. A key inventory audit — comparing the secondary HSM's key list against the primary's last audit log — is the first cryptographic action after HSM failover.

Certificate state after restore. Kubernetes cluster certificates and application TLS certificates have expiry dates that advance regardless of whether the system is running. A cluster restored from a backup that is 30 days old is restored into a state where 30 days have been consumed from every certificate's remaining validity. If any certificate was within 30 days of expiry at backup time, it is expired in the restored cluster. The certificate audit procedure:

# Audit all Kubernetes control-plane certificate expiry
kubeadm certs check-expiration

# Renew expired or near-expiry control-plane certificates
kubeadm certs renew all

# For cert-manager application certificates: force re-issuance
# by deleting Certificate resources and letting cert-manager re-issue
kubectl get certificates -A -o json | \
  jq -r '.items[] | select(.status.notAfter < now | todate) |
         "\(.metadata.namespace)/\(.metadata.name)"' | \
  xargs -I{} kubectl delete certificate -n $(echo {} | cut -d/ -f1) \
                                           $(echo {} | cut -d/ -f2)

# Verify cert-manager issues new certificates
kubectl get certificaterequests -A --watch

Session key re-establishment. Session keys — the ephemeral symmetric keys negotiated during TLS handshakes and encrypted channel establishments — are never stored in the HSM and are never backed up. They exist only in the memory of the communicating processes. After a recovery that restores a system from backup, all active sessions from the backup snapshot are gone; the restored system has no session state. Connected systems — other cluster nodes, remote sensors, C2 peers — will attempt to re-establish sessions using the restored system's long-term credentials (certificates and HSM-backed keys). If those credentials are current and valid, session re-establishment is automatic and transparent. If they are stale or expired, session re-establishment fails and each connection must be manually re-initialized after the credential issue is resolved.

Re-keying procedures post-recovery. For systems where the recovery event itself is treated as a potential key compromise indicator — particularly if the failure was caused by a security incident rather than a hardware or power failure — the ISSO may require a full re-keying cycle before the system returns to classified operations. Re-keying involves generating new KEKs in the restored HSM, re-encrypting all data DEKs under the new KEK, and distributing new certificates to all connected systems. This is a lengthy process that must be budgeted within the recovery timeline if there is any possibility it will be required. Planning documents should explicitly address the re-keying vs. resume-on-existing-keys decision and define the criteria for each path.

The intersection of backup engineering, key management, and Kubernetes operations that classified cloud DR requires is not served by any single tool or framework. It is built from a combination of platform-level backup tooling (Velero, etcdctl, pg_basebackup), HSM-integrated key management, and operational procedures that have been rehearsed under conditions that approximate real disasters. Programs that invest in the rehearsal cadence — and in the honest After-Action Reports that follow — consistently outperform those that treat DR as a documentation exercise.