Installation Manual - 63 Vault post-promotion soak cleanup plan
Post-promotion Vault R1 soak results and cleanup plan for the unused vault-platform store.
This chapter records the read-only post-promotion soak after
vault.v7.comptech-lab.com was promoted from the old locked Vault to
replacement Vault R1.
Governance
| Field | Value |
|---|---|
| Issue | OP-GF-VAULTRECOVERY-1 / #389 |
| Milestone | Workspace Governance |
| ADR | ADR 0028: Greenfield Vault Replacement After Custody Loss |
| Existing controls | ADR 0016 and ADR 0025 |
Validation Summary
Access path:
local coordinator -> dl385-2 -> gf-ocp-bootstrap-01 -> v7 kubeconfigs
Stable DNS still resolves to R1:
vault.v7.comptech-lab.com -> 30.30.200.35, 30.30.200.36, 30.30.200.37
Resolution was confirmed from:
dl385-2gf-ocp-bootstrap-01- hub ESO controller pod
- spoke ESO controller pod
Explicit Vault health checks:
| IP | Cluster | Result |
|---|---|---|
30.30.200.31 | old Vault | initialized, unsealed, standby |
30.30.200.32 | old Vault | initialized, unsealed, active |
30.30.200.33 | old Vault | initialized, unsealed, standby |
30.30.200.35 | R1 | initialized, unsealed, active |
30.30.200.36 | R1 | initialized, unsealed, standby |
30.30.200.37 | R1 | initialized, unsealed, standby |
OpenShift state:
| Cluster | OpenShift | Nodes | ClusterOperators |
|---|---|---|---|
hub-dc-v7 | 4.20.18 | 3/3 Ready | steady |
spoke-dc-v7 | 4.20.18 | 6/6 Ready | steady |
Active ExternalSecrets remain Ready / SecretSynced:
| Cluster | Consumer | Store |
|---|---|---|
hub-dc-v7 | ESO smoke | vault-r1-eso-smoke |
hub-dc-v7 | OADP cloud credentials | vault-r1-oadp |
hub-dc-v7 | RHACS TLS/admin material | vault-r1-rhacs |
spoke-dc-v7 | ESO smoke | vault-r1-eso-smoke |
spoke-dc-v7 | OADP cloud credentials | vault-r1-oadp |
spoke-dc-v7 | logging object-store credentials | logging-local |
spoke-dc-v7 | RHACS TLS material | vault-r1-rhacs |
OADP remains healthy:
| Cluster | DPA | BSL | Schedule | Latest scheduled Backup CR |
|---|---|---|---|---|
hub-dc-v7 | Reconciled | Available | Enabled | platform-resource-daily-20260517223546 |
spoke-dc-v7 | Reconciled | Available | Enabled | platform-resource-daily-20260517224523 |
StackRox remained acceptable on hub and spoke.
Finding
The unused ClusterSecretStore/vault-platform is now invalid on both
clusters:
hub-dc-v7: vault-platform Ready=False / InvalidProviderConfig
spoke-dc-v7: vault-platform Ready=False / InvalidProviderConfig
Argo CD is therefore Synced/Degraded for the applications that own that
object:
hub-dc-v7-bootstrap
spoke-dc-v7-cluster-config
This is not an active secret delivery outage. No live ExternalSecret and no
GitOps-managed ExternalSecret references vault-platform.
Likely cause:
vault-platformstill has the old Vault CA bundle.vault.v7.comptech-lab.comnow resolves to R1.- The R1 serving certificate is issued by the R1 CA.
- The R1 serving certificate includes
vault-r1.v7.comptech-lab.comand30.30.200.35, but notvault.v7.comptech-lab.com.
Cleanup Inventory
DNS:
| Name | A records |
|---|---|
vault.v7.comptech-lab.com | 30.30.200.35, 30.30.200.36, 30.30.200.37 |
gf-ocp-vault-01.v7.comptech-lab.com | 30.30.200.31 |
gf-ocp-vault-02.v7.comptech-lab.com | 30.30.200.32 |
gf-ocp-vault-03.v7.comptech-lab.com | 30.30.200.33 |
vault-r1.v7.comptech-lab.com | no A record |
VM state on dl385-2:
| VM | State |
|---|---|
gf-ocp-vault-seed-01 | running |
gf-ocp-vault-01 | running |
gf-ocp-vault-02 | running |
gf-ocp-vault-03 | running |
gf-ocp-vault-r1-seed-01 | running |
gf-ocp-vault-r1-01 | running |
gf-ocp-vault-r1-02 | running |
gf-ocp-vault-r1-03 | running |
Recommended Next Gate
Remove the unused ClusterSecretStore/vault-platform resources from
hub/spoke GitOps:
clusters/hub-dc-v7/secrets/eso/clustersecretstore-vault.yamlclusters/spoke-dc-v7/secrets/eso/clustersecretstore-vault.yaml
Then validate:
- hub and spoke overlays render;
- server-side dry-run accepts both overlays;
- Argo returns to
Synced/Healthy; - active R1 stores remain Ready /
Valid; - active ExternalSecrets remain Ready /
SecretSynced; - OADP and RHACS remain healthy.
This is lower risk than rotating the R1 serving certificate because no active
consumer uses vault-platform.
Do not decommission old Vault VMs yet. Keep old node-specific DNS records until the Argo degradation is cleared and at least one more scheduled backup window remains healthy.
Actions Not Taken
- No Vault secret, policy, auth role, auth mount, or token was changed.
- No DNS record was changed.
- No VM was stopped or modified.
- No GitOps desired state was changed in this gate.
- No secret values were printed.