2026-05-09 — v6 Fleet Install Day

The single biggest operational day on the v6 fleet — hub-dc-v6 and spoke-dc-v6 both installed, ODF data plane brought up on the spoke, and the management pull-model baselined. The implicit day-wrap that sits between the pre-v6 reset and the day-2 wave.

2026-05-09 is the single most consequential operational day in the v6 fleet’s lifetime. Both clusters — hub-dc-v6 and spoke-dc-v6 — went from agent ISOs on ocp-bootstrap to oc get nodes green inside ~16 hours. The day did not get an explicit “day wrap” GitHub issue the way 2026-05-10 did under #159; the implicit day-wrap is the mid-day session-closeout report at opp-full-plat/reports/sessions/20260509-112323-session-closeout.md, plus the chain of ~30 per-session reports the day produced.

This page collects what happened that day, what shape the fleet took, and where to pick up the thread.

Fleet shape that emerged

Reading the diagram:

  • Hub (left) is the compact 3-AIO management cluster — three nodes, each carrying control-plane,master,worker roles. No physical workers, no ODF, deliberately storage-light per ADR 0018.
  • Spoke (right) is the workload cluster — three VM masters plus three physical worker boxes (gold-1, gold-2, gpu-01), with the physical workers running on the 30.30.0.0/16 NIC ports after the first MAC selection picked the wrong path.
  • Green dashed line is the ACM pull model: the spoke registers to the hub, the hub manages the spoke through ACM + per-cluster Argo, and platform-gitops is the source of truth on both sides.

The hub install completed at 08:00 UTC; the spoke base install completed at 15:20 UTC. Everything else on this day was the work needed to get those two milestones across the line, plus the immediate day-2 baseline that turned the installed clusters into a fleet.

The implicit day-wrap

reports/sessions/20260509-112323-session-closeout.md (closed at 11:23 UTC, mid-day) is the day’s only explicit closeout artifact. It scoped the morning:

ItemWhat
Scopedeveloper readiness + Docker runtime handoff; federated GitOps ADR/milestones/gates + GitLab FG-1 execution; workspace submodule cleanup; spoke-dc-v6 install preflight
Key decisionsADR 0014 (developer-readiness contract); ADR 0015 (federated GitOps repo architecture); Gate H1 added; FG-1 GitLab group skeleton
Follow-ups carried forward#83 FG-1 (open until CODEOWNERS, CE-compatible merge controls, validation pipelines, runner posture, negative access tests); #67 developer-readiness gate (open until sample app/template, Trivy/MinIO evidence, secret custody, OpenShift namespace onboarding, end-to-end smoke evidence)
Closing instructionResume with the spoke-dc-v6 install-input package; do not boot anything until inputs reviewed and the user explicitly approves

The user did approve, mid-afternoon, and the spoke went up. The afternoon’s work isn’t covered by that closeout — it’s covered by the per-session report chain.

Per-session report chain — what happened, in order

Times are UTC. Slugs map to files under opp-full-plat/reports/sessions/20260509-<HHMMSS>-<slug>.md.

Morning — hub install + image supply

TimeSlugResult
06:37oc-mirror-partial-completeDay-prior oc-mirror partial completion captured; sets up the disconnected install with mirrored release + operator content
06:45hub-dc-v6-install-preflightPreflight checks: pull-secret, IDMS, ITMS, agent install workdir all staged on ocp-bootstrap
06:53hub-dc-v6-install-workdir-prepInstall workdir at /home/ze/ocp-clusters/hub-dc-v6 staged
07:11hub-dc-v6-iso-vm-definitionsAgent ISO generated + libvirt VM XMLs for hub-dc-v6-master-{0,1,2}
08:00hub-dc-v6-boot-install-completeHub install complete. OCP 4.20.18, all three masters Ready control-plane,master,worker, all ClusterOperators clean, DNS resolves api/api-int/apps to the planned VIPs 30.30.75.10/.11
08:15hub-dc-v6-local-connection-cleanupLocal-only auth/ output handled on ocp-bootstrap; nothing committed
08:23hub-dc-v6-day1-checkpoint-baselineDay-1 baseline recorded
08:43hub-dc-v6-disconnected-catalog-baselineIDMS/ITMS applied; mirrored catalogs added; default CatalogSource set disabled; master MCP rolled once before pulls worked correctly

The hub install itself was the easy part — the real load-bearing piece of work was the catalog baseline at 08:43, which is what made the hub usable for operator installs. That manifest set is exactly what REPRO-HUB-1 later adopted into platform-gitops (see the dedicated REPRO-HUB-1 page).

Mid-morning — developer readiness + federated GitOps ADRs

The morning split into two threads. While the hub was rolling its catalog baseline, a second thread set up the developer-readiness and federated-GitOps contracts:

TimeSlugResult
08:55developer-readiness-trackADR 0014 (developer-readiness contract), milestone #23, issue #67
08:57hub-dc-v6-openshift-gitops-installOpenShift GitOps operator v1.20.3 installed
09:01developer-handbook-scaffoldmdBook-compatible developer handbook scaffolded
09:12hub-dc-v6-minimal-gitlab-gitops-bootstrapFirst Argo Application/hub-dc-v6-bootstrap Synced/Healthy
09:14developer-handbook-runtime-scopeHandbook expanded to include the Docker-runtime path
09:28hub-dc-v6-gitops-appproject-hardeningAppProject hardening on the hub
09:34docker-runtime-vmdocker-runtime-vm deployed as the alternative app-delivery target
09:39hub-dc-v6-bootstrap-namespace-baselineBootstrap namespace baseline recorded
10:08federated-gitops-adr-and-milestonesADR 0015 (federated GitOps repo architecture) + milestones

By 10:08 the hub was Argo-bootstrapped against platform-gitops and the federated-GitOps contract was written. From this point, every subsequent operator install on either cluster ships through a GitLab MR.

Mid-day — spoke install package + workspace governance

TimeSlugResult
11:17spoke-dc-v6-install-preflightRead-only preflight
11:23session-closeoutImplicit day-wrap. Morning scope closed; user gate set on spoke boot
11:31spoke-dc-v6-install-inputsSpoke install inputs prepared
11:54spoke-dc-v6-install-workdir-prepSpoke workdir staged
13:02spoke-dc-v6-iso-vm-definitionsSpoke agent ISO + VM definitions
13:25spoke-dc-v6-physical-worker-gatePhysical worker boot safety gate — explicit checklist before any iLO action
14:14parallel-agent-worktree-isolationParallel-agent worktree isolation policy → ADR 0017 (the cost of NOT having it was a misleading commit subject on origin earlier in the week)

Afternoon — spoke base install + ODF disk reset

TimeSlugResult
15:20spoke-dc-v6-install-completeSpoke base install complete. OCP 4.20.18, 3 VM masters (30.30.75.33/34/35) + 3 physical workers (30.30.75.36/37/38) all Ready. Physical-worker MACs corrected to ens1f1np1 (gold-1/gold-2) and ens15f3 (gpu-01); iLO virtual USB media used instead of CD/DVD because byte-range behavior matters; virtual media ejected before installed-RHCOS reboot
15:34spoke-dc-v6-odf-preflight-blockedODF preflight blocked on disk inventory — devices not safe to wipe without review
15:52spoke-dc-v6-odf-disk-remediationDisk remediation pass
16:16spoke-dc-v6-odf-lso-gitops-drift-auditLSO drift audit — identified manual residue from the failed first attempt
16:27spoke-dc-v6-manual-odf-lso-removalManual ODF + LSO removal (full reset before a clean GitOps-driven install)
16:32hub-dc-v6-management-gitops-resetHub management GitOps reset — clean slate for the pull-model bootstrap
16:49management-gitops-pull-model-baselineADR 0018 — pull-model baseline (ACM + per-cluster Argo on the spoke)

The spoke base install itself was clean. ODF was the painful part — the disks weren’t safe to wipe without a review pass, the first attempt left residue, and the cleanup ran across 16:16-16:27 before the cluster was ready for a GitOps-driven ODF install.

Evening — ACM registration + Nexus image-supply baseline

TimeSlugResult
20:10spoke-dc-v6-acm-registrationspoke-dc-v6 ACM-registered to hub
20:15argo-orphan-warning-reviewArgo orphan-warning review
20:42acm-openshift-gitops-pull-starterACM + OpenShift GitOps pull starter — the gitops-addon path
22:13spoke-dc-v6-odf-storage-consumer-cleanupODF StorageConsumer cleanup
22:47spoke-dc-v6-odf-csi-mirror-fixODF CSI image mirror fix (#120)
23:27nexus-image-supply-baselineADR 0019 — Nexus-only image supply baseline

By 23:27 the fleet had the foundational guardrails written down: ADR 0014 (developer-readiness), ADR 0015 (federated GitOps), ADR 0017 (parallel-agent worktree isolation), ADR 0018 (pull-model baseline), ADR 0019 (Nexus-only image supply). Every operator install that ships after this day is bound by them.

What this day produced

OutputWhere it lives today
hub-dc-v6 clusterlive; compact 3-AIO; ACM hub + OpenShift GitOps + day-1 hub catalog adopted into platform-gitops later under REPRO-HUB-1
spoke-dc-v6 clusterlive; 3 VM + 3 phys workers; ODF data plane (after the manual reset); ACM-registered to hub
OpenShift GitOps bootstrap on hubfirst Application/hub-dc-v6-bootstrap Synced/Healthy at 09:12 UTC
OpenShift GitOps bootstrap on spokewired into ACM pull model by 20:42 UTC
ADR 0014developer-readiness platform contract
ADR 0015federated GitOps repo architecture
ADR 0017parallel-agent worktree isolation (after a real incident earlier in the week)
ADR 0018management-GitOps pull-model baseline
ADR 0019Nexus-only image supply baseline (three-endpoint split: install / dev pull / app push)
GitLab FG-1 group skeletonissue #83 follow-ups carried forward
Developer handbook scaffoldmdBook-compatible, includes Docker-runtime path
Issues opened#93 (spoke install), #120 (ODF CSI image mirror), milestone #28 (spoke install), milestone #23 (developer readiness)

How this day relates to the rest of the history

This day sits between two other entries in the history-and-replay section:

  • Upstream: pre-v6 → v6 transition. The pre-v6 fleet (hub-dc, spoke-dc, hub-dr, spoke-dr under ADR 0001) was retired; ADR 0022 supersedes ADR 0001 and reserves the v6 names. 2026-05-09 is the day the active half of that supersession became physical clusters.
  • Downstream: the day-2 wave. Everything after 2026-05-09 — operator installs, PCI-DSS chain, ODF storage-consumer cleanup, ACM gitops-addon Routes-CRD incident, REPRO-HUB-1 adoption — runs against the clusters this day produced. The Day-Wrap Archive carries the per-day record from 2026-05-10 onward.

Why the implicit day-wrap was enough

A formal day-wrap GitHub issue (the way 2026-05-10 got #159) wasn’t needed for this day. The two install milestones each have their own session-closeout records (hub-dc-v6-boot-install-complete, spoke-dc-v6-install-complete) with the actual Install complete lines, the oc get nodes output, the residual risks, and the next-step gates. The mid-day session-closeout covers the morning’s ADR + milestone work. The afternoon and evening run was per-session reports plus SESSION_LOG.md. Together they produce the same audit trail as a day-wrap roll-up, with less duplication.

The convention going forward (set on 2026-05-05 18:39 UTC under session-routines.md): per-session reports are mandatory for any non-trivial session; day-wrap issues are reserved for days that produce enough roll-up-able churn that the per-session reports alone leave gaps. 2026-05-09 didn’t have those gaps — it had two crisp install completes and the contract ADRs to bind them.

References

  • Session reports (UTC times): all under opp-full-plat/reports/sessions/20260509-<HHMMSS>-<slug>.md
  • Day’s implicit closeout: opp-full-plat/reports/sessions/20260509-112323-session-closeout.md
  • Hub install completion: opp-full-plat/reports/sessions/20260509-080024-hub-dc-v6-boot-install-complete.md
  • Spoke install completion: opp-full-plat/reports/sessions/20260509-152004-spoke-dc-v6-install-complete.md
  • Hub catalog baseline (becomes REPRO-HUB-1 input): opp-full-plat/reports/sessions/20260509-084300-hub-dc-v6-disconnected-catalog-baseline.md
  • Pre-v6 to v6 transition — what came before
  • REPRO-HUB-1 hub adoption — how the hub day-1 baseline became GitOps after the fact
  • Day-Wrap Archive — the per-day record from 2026-05-10 onward
  • ADRs accepted this day: 0014 (developer-readiness), 0015 (federated GitOps), 0017 (parallel-agent worktree isolation), 0018 (pull-model baseline), 0019 (Nexus-only image supply)

Last reviewed: 2026-05-12