Installation Manual - 116 hub-dr-v7 deployment program plan

The milestone, workstream, phases, validation matrix, and reproducibility plan for the hub-dr-v7 warm standby deployment.

This chapter records the deployment program for hub-dr-v7, before live deployment work begins.

The goal is a fully validated, documented, reproducible warm standby management cluster for hub-dc-v7.

Governance

FieldValue
WorkstreamOP-GF-HUBDRV7
Milestonehub-dr-v7 Warm Standby Deployment
Parent issue#448 / OP-GF-HUBDRV7-0
Completed design issue#446 / OP-GF-HUBDRV7-1
Active next issue#451 / OP-GF-HUBDRV7-5
Governing ADRsADR 0016, ADR 0017, ADR 0024, ADR 0025, ADR 0029
Local planplans/hub-dr-v7-warm-standby-deployment-program.md

Certification Boundary

In this workstream, certified means internally certified for the CompTech lab:

  • every phase has evidence;
  • every live change has an issue and rollback path;
  • every important state is reproducible from recorded inputs;
  • OpenShift health, GitOps health, operator health, backup readiness, compliance scans, security checks, and failover readiness are validated;
  • documentation exists locally and in this public installation manual.

This is not an external Red Hat certification, PCI DSS ROC/AOC, QSA, ISO, or bank-auditor attestation. External attestation requires a separate auditor-led track.

Non-negotiable Controls

  • hub-dr-v7 stays warm standby until a failover issue explicitly promotes it.
  • Do not clone hub-dc-v7 identity: no copied etcd, cluster ID, kubeconfigs, OAuth secrets, service-account signers, RHACS runtime secrets, or ACM runtime state.
  • No live mutation without a phase issue, milestone, ADR coverage, validation plan, rollback path, and closeout.
  • Do not print or commit secrets, kubeconfigs, pull secrets, PATs, Vault tokens, private keys, passwords, or full Secret manifests.
  • Durable OpenShift state must be GitOps-first.
  • Every material cluster gate must update this installation manual before closeout.

Target State

AreaTarget
RoleWarm standby management hub
TopologyCompact 3-node VM-only cluster
ReleaseMatch hub-dc-v7, currently OpenShift 4.20.18
Node rolecontrol-plane,master,worker
VM spec32 vCPU, 64 GiB RAM per master
VM disks300 GiB boot + 3 x 500 GiB data per master
Candidate API/API-int VIP30.30.220.10
Candidate ingress VIP30.30.220.11
Candidate masters30.30.220.13-15
Candidate MACs52:54:00:23:00:00-02
Cluster network10.128.0.0/14, host prefix 23
Service network172.30.0.0/16

Phase Line

The issue line remains granular for auditability. Execution and public documentation from #451 onward should be batched so each article reads as a complete runbook.

PhaseIssueStatusGate
OP-GF-HUBDRV7-0#448OpenProgram ownership and certification evidence plan
OP-GF-HUBDRV7-1#446ClosedDesign and read-only preflight
OP-GF-HUBDRV7-2#447ClosedRender install inputs and edge plan
OP-GF-HUBDRV7-3#449ClosedReserve IPAM, DNS, and HAProxy edge plan
OP-GF-HUBDRV7-4#450ClosedPrepare VM env and libvirt dry-run
OP-GF-HUBDRV7-5#451OpenRender agent ISO and create VMs
OP-GF-HUBDRV7-6#452OpenValidate base OpenShift install
OP-GF-HUBDRV7-7#453OpenBootstrap GitOps, catalogs, mirrors, and storage foundation
OP-GF-HUBDRV7-8#454OpenInstall management operators in standby mode
OP-GF-HUBDRV7-9#455OpenProve backup, restore, and secret custody
OP-GF-HUBDRV7-10#456OpenClose compliance and hardening evidence
OP-GF-HUBDRV7-11#457OpenOperational readiness and SLO validation
OP-GF-HUBDRV7-12#458OpenWrite and dry-run failover runbook
OP-GF-HUBDRV7-13#459OpenControlled takeover drill
OP-GF-HUBDRV7-14#460OpenReproducibility certification evidence and closeout
OP-GF-HUBDRV7-15#461OpenFull installation manual and handoff

Batch Execution Model

The remaining work is grouped into four execution batches. Each batch may close multiple child issues, but the public installation manual should get one comprehensive article per batch.

BatchIssuesWhat It Proves
Prep batch, completed#446, #447, #449, #450Design, install inputs, edge DNS, access, and VM dry-run readiness
Batch 1: Build and base cluster#451, #452The VMs boot, the agent install completes, and base OpenShift health is clean
Batch 2: Standby platform foundation#453, #454GitOps, catalogs, mirrors, storage, cert-manager, ESO/Vault auth, ACM/MCE, and RHACS are healthy without taking over spoke-dc-v7
Batch 3: Protection and readiness#455, #456, #457Backup/restore, secret custody, compliance, hardening, monitoring, alerting, logging, and SLO checks are recorded
Batch 4: DR certification and handoff#458, #459, #460, #461Failover runbook, controlled takeover drill, reproducibility evidence, and full handoff are complete

Future articles should include objective, non-goals, entry criteria, execution sequence, rollback, validation matrix, secrets handling, GitOps/live deltas, residual risks, and the next-batch decision.

Gate Model

Every phase must close with:

  • issue comment with what changed;
  • validation evidence;
  • files touched;
  • live systems touched;
  • rollback or promotion decision;
  • residual risks;
  • next gate;
  • local session record;
  • updated local memory files;
  • blog chapter or blog update when the gate is material.

Validation Matrix

DomainRequired evidence
GovernanceIssue, milestone, phase, ADR coverage
NetworkIPAM, DNS, HAProxy validation and rollback
VMLibvirt domain, CPU, RAM, disk, NIC, boot state
Install inputsinstall-config, agent-config, mirror resources, FIPS, host count
Base clusterClusterVersion, nodes, MCPs, ClusterOperators, CSRs, pods
GitOpsRoot app, revision, sync, health, render validation
OperatorsSubscription, InstallPlan, CSV, operand readiness
SecretsVault/ESO path and readiness, no value disclosure
BackupDPA, BSL, schedule, backup, restore or dry-run evidence
SecurityRHACS health, Gatekeeper health, NetworkPolicies, image policy
ComplianceCIS, PCI DSS 4, File Integrity, exception register
OperationsAPI, console, ingress, monitoring, logging, alerting
DRFailover runbook, dry-run, takeover drill, rollback
DocumentationLocal report, session log, runbook, blog chapter
ReproducibilitySource paths, commits, commands, artifacts, rebuild notes

Final Done Criteria

hub-dr-v7 is complete only when all are true:

  1. cluster is installed and healthy;
  2. platform GitOps is synced and healthy;
  3. management services are installed in standby-safe mode;
  4. backup, restore readiness, and secret custody are proven;
  5. compliance and hardening evidence is closed or explicitly excepted;
  6. operational readiness checks pass;
  7. failover runbook dry-run is complete;
  8. controlled takeover drill is completed or explicitly deferred;
  9. public installation manual and local runbooks are complete;
  10. reproducibility evidence pack is complete.

Next Gate

The next execution unit is Batch 1:

#451 / OP-GF-HUBDRV7-5: render hub-dr-v7 agent ISO and create VMs
#452 / OP-GF-HUBDRV7-6: validate base hub-dr-v7 OpenShift install

The closeout should be one comprehensive build and base-install article, not two thin gate notes.

Last reviewed: 2026-05-21