Installation Manual - 60 Vault R1 RHACS migration

Migration of hub-dc-v7 and spoke-dc-v7 RHACS ExternalSecrets to replacement Vault R1.

This chapter records the RHACS consumer migration to replacement Vault R1. It followed the ESO smoke gate and moved only RHACS ExternalSecrets to a dedicated R1 store.

Governance

FieldValue
IssueOP-GF-VAULTRECOVERY-1 / #389
MilestoneWorkspace Governance
ADRADR 0028: Greenfield Vault Replacement After Custody Loss
Existing controlsADR 0016 and ADR 0025

Preflight

Read-only checks used the required path:

local coordinator -> dl385-2 -> gf-ocp-bootstrap-01 -> v7 kubeconfigs
ClusterOpenShiftClusterOperatorsNodesRHACS ExternalSecretsOADP
hub-dc-v74.20.18steady3/3 ReadyReady on vault-platformDPA Reconciled, BSL Available, Velero 1/1
spoke-dc-v74.20.18steady6/6 ReadyReady on vault-platformDPA Reconciled, BSL Available, Velero 1/1

Live RHACS target Secret key shapes were checked without printing values.

Replacement Vault Seed

Replacement Vault R1 was seeded from the currently reconciled Kubernetes Secrets. Secret values were not printed.

Created replacement Vault roles:

ClusterAuth mountRolePolicy
hub-dc-v7kubernetes-hub-dc-v7rhacs-secretshub-dc-v7-rhacs-secrets
spoke-dc-v7kubernetes-spoke-dc-v7rhacs-secretsspoke-dc-v7-rhacs-secrets

Seeded replacement Vault paths:

PathKey shape
secret/greenfield/openshift/hub-dc-v7/rhacs/init-bundleadmission-control, collector, and sensor TLS properties
secret/greenfield/openshift/hub-dc-v7/rhacs/adminpassword
secret/greenfield/openshift/spoke-dc-v7/rhacs/init-bundleadmission-control, collector, and sensor TLS properties

Kubernetes auth read validation passed for the rhacs-secrets role on both clusters using short-lived service-account tokens. The validation printed only property keys.

GitOps Change

GitOps commit:

93daa29 Move RHACS secrets to Vault R1

The commit added:

ClusterNew store
hub-dc-v7ClusterSecretStore/vault-r1-rhacs
spoke-dc-v7ClusterSecretStore/vault-r1-rhacs

Then RHACS ExternalSecrets were changed to reference vault-r1-rhacs.

Hub moved:

  • stackrox/admission-control-tls
  • stackrox/central-admin-password
  • stackrox/collector-tls
  • stackrox/sensor-tls

Spoke moved:

  • stackrox/admission-control-tls
  • stackrox/collector-tls
  • stackrox/sensor-tls

Validation

Both overlays rendered locally with kubectl kustomize, and server-side dry-run accepted both overlays. The dry-run showed that ClusterSecretStore/vault-r1-rhacs would be created on each cluster.

The bootstrap GitOps clone was fast-forwarded to 93daa29, and Argo CD was hard-refreshed.

Argo applicationSyncHealthRevision
hub-dc-v7-bootstrapSyncedHealthy93daa29
hub-side spoke-dc-v7-cluster-configSyncedHealthy93daa29
spoke-local spoke-dc-v7-cluster-configSyncedHealthy93daa29

Post-change RHACS validation:

ClusterStoreExternalSecretsRefresh timesTarget Secret keys
hub-dc-v7vault-r1-rhacs Ready4/4 Ready / SecretSynced2026-05-17T23:23:33Z to 2026-05-17T23:23:35Zexpected admin and TLS keys
spoke-dc-v7vault-r1-rhacs Ready3/3 Ready / SecretSynced2026-05-17T23:23:22Zexpected TLS keys

Other consumers stayed Ready:

ConsumerStoreResult
ESO smokevault-r1-eso-smokeReady
OADP cloud credentialsvault-r1-oadpReady
Spoke logging bridgelogging-localReady

Final State

ClusterOpenShiftClusterOperatorsNodesStackRox podsOADP
hub-dc-v74.20.18steady3/3 Ready18/18 acceptableDPA Reconciled, BSL Available, Velero 1/1
spoke-dc-v74.20.18steady6/6 Ready16/16 acceptableDPA Reconciled, BSL Available, Velero 1/1

Actions Not Taken

  • No old Vault mutation was made.
  • No stable Vault DNS cutover was made.
  • No RHACS certificate or admin password rotation was performed.
  • No secret values were printed.

Next Action

Run a final stable Vault DNS promotion readiness gate before changing vault.v7.comptech-lab.com.