Installation Manual - 92 Operator readiness and batch execution

Read-only operator readiness, parity audit, install/defer decisions, the consolidated v7 operator batch plan, and the spoke Pipelines, observability foundation, and tracing/mesh executions.

This chapter records the OP-GF-OPERATORS-01 read-only operator-readiness gate for the greenfield v7 clusters. It also carries the OP-GF-OPERATORS-18 v6-to-v7 operator parity addendum, the OP-GF-OPERATORS-19 missing-operator decision, and the OP-GF-OPERATORS-20 batch model.

The completed 2026-05-21 batch closeouts are also published as standalone installation-manual chapters so they are visible from the Greenfield manual index without splitting each batch into separate preflight, installation, validation, and rollback pages:

The initial readiness gate did not change live cluster state. Later batch sections state their live changes explicitly.

Governance

Access Path

All live checks used the established path:

local workspace -> dl385-2 -> gf-ocp-bootstrap-01 -> v7 kubeconfigs

Kubeconfigs on gf-ocp-bootstrap-01:

/home/ze/ocp-greenfield-deployment/artifacts/openshift/hub-dc-v7/auth/kubeconfig
/home/ze/ocp-greenfield-deployment/artifacts/openshift/spoke-dc-v7/auth/kubeconfig

Desired State

The bootstrap GitOps clone was clean and matched the live Argo revision:

repo: /home/ze/greenfield-ops/openshift-gitops
HEAD: 12c68aee6d6a77dfcc197926d4f36594ea224625
status: main...origin/main

Cluster Health

Installed Operators

All installed Subscriptions reported AtLatestKnown. No non-complete InstallPlans were found.

Hub operator areas:

Advanced Cluster Management
cert-manager
Compliance Operator
External Secrets Operator
LVMS
Loki Operator
MCE
OADP
OpenShift GitOps
OpenShift Logging
RHACS

Spoke operator areas:

cert-manager
Compliance Operator
External Secrets Operator
File Integrity Operator
Local Storage Operator
Loki Operator
OADP
ODF dependency operators
OpenShift Logging
RHACS secured cluster

Platform Dependencies

OADP is installed and reconciled on both clusters.

External Secrets stores are Ready/Valid, and no non-ready ExternalSecrets were reported.

Spoke ODF/Ceph/NooBaa, Loki, log forwarding, and RHACS secured cluster are steady.

Compliance State

Hub Compliance Operator state:

ocp4-cis: DONE / COMPLIANT
ocp4-cis-node-master: DONE / COMPLIANT
counts: PASS=162, MANUAL=21, FAIL=0

Spoke Compliance Operator state remains partially non-compliant for known unrelated controls:

PASS=1028
FAIL=432
MANUAL=107

Catalog Finding

The platform is healthy enough for more work, but catalog hygiene is not ready for another operator install.

Both clusters still have:

OperatorHub/cluster spec: {}
default classic sources enabled
external classic CatalogSources READY
external OLM v1 ClusterCatalogs Serving=True

Hub has mirrored OLM v1 cc-* ClusterCatalogs:

cc-certified-operator-index-v4-20
cc-redhat-operator-index-v4-20

Spoke does not currently show mirrored cc-* ClusterCatalogs.

Installed Subscriptions are using mirrored classic cs-* sources, but future PackageManifest resolution is mixed while the external defaults remain available.

Decision

Do not install another operator as the immediate next step.

Selected next gate:

OP-GF-OPERATORS-02: disconnected catalog hygiene and package-source pinning

That gate should make catalog resolution mirror-only and deterministic before another operator install.

First Operator Candidate

After catalog hygiene, the first actual operator candidate is:

Gatekeeper preflight and dry-run-only install design

Gatekeeper is a reasonable governance/security next layer, but it affects admission. It should wait until catalog source resolution is clean and the install plan is dry-run reviewed.

2026-05-21 Parity Addendum

OP-GF-OPERATORS-18 audited whether v7 has the operator shape planned from the retired v6 environment. v6 was not queried live because it is gone; the audit used historical local records for v6 and live read-only checks for hub-dc-v7 and spoke-dc-v7.

No live object, GitOps desired state, VM service, DNS record, HAProxy config, or secret was changed.

Live v7 health at the audit point:

Core parity is substantially present:

The audit also confirmed that not every first-mirror or catalog-visible operator package is installed in v7. Missing or deferred decision items are:

OpenShift Pipelines
Cluster Observability Operator / ACM observability operand
NetObserv
Tempo
OpenTelemetry
Service Mesh / Kiali / Jaeger
Quay Operator
Container Security Operator
Open Liberty Operator
Kube Descheduler
Security Profiles Operator
CloudNativePG package for application database use
NFD / NVIDIA GPU Operator
OpenShift Virtualization
Serverless
OpenShift AI / RHODS

The result is not an automatic install queue. The next operator gate should decide which missing families are required now, which stay mirrored but deferred, which are replaced by external services, and which should be removed from the active expectation.

Recommended next gate:

OP-GF-OPERATORS-19: v7 missing-operator decision and install/defer plan

2026-05-21 Missing-operator Decision

OP-GF-OPERATORS-19 converted the missing/deferred list into a concrete install/defer plan. No operators were installed or removed in that gate.

The only immediate next install candidate is:

OP-GF-OPERATORS-20: spoke OpenShift Pipelines operator-only install

Guardrails for that future gate:

Deferred tracks:

Removed or replaced active parity expectations:

Kept out of active scope until explicitly reintroduced:

NFD / NVIDIA GPU Operator
OpenShift AI / RHODS
OpenShift Virtualization
Serverless

2026-05-21 Operator Batch Execution Model

OP-GF-OPERATORS-20 changed the remaining operator work into consolidated batches. Each batch now carries its own preflight, GitOps install, validation, rollback or promotion decision, and documentation closeout.

The previous narrow next label, OP-GF-OPERATORS-20: spoke OpenShift Pipelines operator-only install, is superseded by the batch model. The Pipelines install remains the next actual operator action, but it moves to Batch 1:

OP-GF-OPERATORS-21: Batch 1 app delivery foundation - spoke OpenShift Pipelines operator-only

Every operator batch must include:

Planned batches:

Do not include these in the active batch queue unless their domain is explicitly reintroduced:

Quay Operator
Container Security Operator
NFD / NVIDIA GPU Operator
OpenShift AI / RHODS
OpenShift Virtualization
Serverless

2026-05-21 Batch 1: spoke OpenShift Pipelines

OP-GF-OPERATORS-21 installed the Red Hat OpenShift Pipelines operator on spoke-dc-v7 as the first consolidated operator batch.

Governance:

Scope:

The batch remained operator-only from a user workload perspective. It did not create application namespaces, CI/CD credentials, build workloads, PipelineRuns, TaskRuns, TriggerBindings, TriggerTemplates, or EventListeners.

Batch preflight

Preflight used the standard path:

local workspace -> dl385-2 -> gf-ocp-bootstrap-01 -> spoke-dc-v7 kubeconfig

Readiness checks passed before install:

The PackageManifest resolved from the mirrored Red Hat catalog with default channel pipelines-1.22 and current CSV openshift-pipelines-operator-rh.v1.22.0. The package supports AllNamespaces install mode.

Related image source prefixes identified for mirror policy:

registry.redhat.io/openshift-pipelines
registry.redhat.io/openshift-serverless-1
registry.redhat.io/rhel9
registry.redhat.io/source-to-image
registry.redhat.io/ubi9

GitOps install

Platform GitOps commit 8c41a86 added:

clusters/spoke-dc-v7/operators/openshift-pipelines-operator/kustomization.yaml
clusters/spoke-dc-v7/operators/openshift-pipelines-operator/subscription.yaml
clusters/spoke-dc-v7/platform/image-mirrors/idms-openshift-pipelines.yaml

It also updated the spoke overlay kustomizations and the platform GitOps CHANGELOG.md.

The Subscription lives in openshift-operators, uses source cs-redhat-operator-index-v4-20, channel pipelines-1.22, and starting CSV openshift-pipelines-operator-rh.v1.22.0.

ImageDigestMirrorSet/idms-openshift-pipelines maps the related Red Hat source prefixes to quay.v7.comptech-lab.com/openshift-operators/*.

GitOps render and server-side dry-run passed before push. The bootstrap clone on gf-ocp-bootstrap-01 was then fast-forwarded to 8c41a86.

Validation

Final validation:

All TektonInstallerSet objects reported Ready.

Scope caveat

The operator creates default operator-owned Tekton add-ons. At validation time those included default Task and Pipeline resources owned by TektonInstallerSet objects.

The batch still created no workload runs or trigger entry points:

pipelineruns.tekton.dev: 0
taskruns.tekton.dev: 0
triggerbindings.triggers.tekton.dev: 0
triggertemplates.triggers.tekton.dev: 0
eventlisteners.triggers.tekton.dev: 0

Treat the default Tasks/Pipelines as operator-managed add-ons, not tenant application delivery onboarding.

Promotion decision

Keep the install.

The install is promoted because GitOps is synced, OLM is AtLatestKnown, the CSV is Succeeded, TektonConfig is Ready, operator pods are Running, and spoke cluster health stayed steady.

Rollback, if needed, is GitOps-first:

1. Remove operators/openshift-pipelines-operator from the spoke overlay.
2. Remove idms-openshift-pipelines only if no other installed workload depends on those mirror mappings.
3. Sync and validate OLM/operator pruning.
4. Do not force-delete Tekton CRDs, the operator-created namespace, or cluster-scoped Tekton objects without a separate cleanup issue.

Next operator batch:

OP-GF-OPERATORS-22: Batch 2 observability foundation - Cluster Observability and NetObserv

2026-05-21 Batch 2: spoke observability foundation operators

OP-GF-OPERATORS-22 installed the Cluster Observability Operator and NetObserv Operator on spoke-dc-v7 as the second consolidated operator batch.

Governance:

Scope:

The batch intentionally stopped at operator readiness. It did not create FlowCollector, FlowMetric, MonitoringStack, UIPlugin, ObservabilityInstaller, Perses, PersesDashboard, or MultiClusterObservability resources. It also did not introduce object storage, retention policy, flow collection, telemetry tenants, sample applications, tracing, service mesh, or workload onboarding.

Batch preflight

Preflight used the standard path:

local workspace -> dl385-2 -> gf-ocp-bootstrap-01 -> spoke-dc-v7 kubeconfig

Readiness checks passed before install:

The PackageManifests resolved from the mirrored Red Hat catalog:

Related image source prefixes identified for spoke mirror policy:

registry.redhat.io/cluster-observability-operator
registry.redhat.io/network-observability
registry.redhat.io/openshift-logging

The hub already had broad mirror mappings for the observability source prefixes. The spoke did not, so Batch 2 added spoke-only mirror policy.

GitOps install

Platform GitOps commit dd6d1da added:

clusters/spoke-dc-v7/operators/cluster-observability-operator/
clusters/spoke-dc-v7/operators/netobserv-operator/
clusters/spoke-dc-v7/platform/image-mirrors/idms-observability-foundation.yaml

It also updated the spoke overlay kustomizations and the platform GitOps CHANGELOG.md.

The new namespaces use sync wave 5. The OperatorGroups and Subscriptions use sync wave 10.

ImageDigestMirrorSet/idms-observability-foundation maps the related Red Hat source prefixes to:

quay.v7.comptech-lab.com/openshift-operators/cluster-observability-operator
quay.v7.comptech-lab.com/openshift-operators/network-observability
quay.v7.comptech-lab.com/openshift-operators/openshift-logging

GitOps render passed. Server-side dry-run passed for the new namespaces and IDMS. A full rendered-stream server-side dry-run hit the expected dry-run limitation where namespaced OperatorGroups and Subscriptions cannot be dry-run created into namespaces that only exist in the same dry-run stream; no live object was changed by that failed full-stream dry-run.

The bootstrap clone on gf-ocp-bootstrap-01 was then fast-forwarded to dd6d1da.

Validation

Final validation:

Operator pods observed Running:

obo-prometheus-operator
obo-prometheus-operator-admission-webhook
observability-operator
perses-operator
netobserv-controller-manager
netobserv-plugin-static

Promotion decision

Keep the install.

The install is promoted because GitOps is synced, OLM is AtLatestKnown, both CSVs are Succeeded, operator pods are Running, no excluded operands were created, and spoke cluster health stayed steady.

Rollback, if needed, is GitOps-first:

1. Remove the two Batch 2 operator directories from the spoke overlay.
2. Remove idms-observability-foundation only if no other installed workload depends on those mirror mappings.
3. Sync and validate OLM/operator pruning.
4. Do not force-delete the newly introduced CRDs or operator namespaces without a separate cleanup issue.

Next operator batch:

OP-GF-OPERATORS-23: Batch 3 tracing and mesh - Tempo, OpenTelemetry, Service Mesh, and Kiali

2026-05-21 Batch 3: spoke tracing and mesh operators

OP-GF-OPERATORS-23 installed the Tempo, OpenTelemetry, Service Mesh 3, and Kiali operators on spoke-dc-v7 as the third consolidated operator batch.

Governance:

Scope:

The batch intentionally stopped at operator readiness. It did not create a TempoStack, TempoMonolithic, OpenTelemetryCollector, OpAMPBridge, TargetAllocator, ServiceMeshControlPlane, ServiceMeshMemberRoll, ServiceMeshMember, Sail Istio, IstioRevision, ztunnel, Kiali, OSSMConsole, Jaeger, sample application, mesh tenant namespace, telemetry pipeline, trace storage, or retention policy.

Batch preflight

Preflight used the standard path:

local workspace -> dl385-2 -> gf-ocp-bootstrap-01 -> spoke-dc-v7 kubeconfig

Readiness checks passed before install:

The PackageManifests resolved from the mirrored Red Hat catalog:

Related image source prefixes identified for spoke mirror policy:

registry.redhat.io/rhosdt
registry.redhat.io/openshift4
registry.redhat.io/openshift-service-mesh
registry.redhat.io/openshift-service-mesh-tech-preview

The hub already had broad mirror mappings for these source prefixes. The spoke did not, so Batch 3 added spoke-only mirror policy.

GitOps install

Platform GitOps commit 0f08a74 added:

clusters/spoke-dc-v7/operators/tempo-operator/
clusters/spoke-dc-v7/operators/opentelemetry-operator/
clusters/spoke-dc-v7/operators/service-mesh-operator/
clusters/spoke-dc-v7/operators/kiali-operator/
clusters/spoke-dc-v7/platform/image-mirrors/idms-tracing-mesh.yaml

It also updated the spoke overlay kustomizations and the platform GitOps CHANGELOG.md.

Tempo and OpenTelemetry use dedicated namespaces because their PackageManifest records advertise suggested namespaces. Service Mesh 3 and Kiali use the existing openshift-operators namespace because the PackageManifest records did not advertise a dedicated namespace and both packages support AllNamespaces install mode.

ImageDigestMirrorSet/idms-tracing-mesh maps the related Red Hat source prefixes to:

quay.v7.comptech-lab.com/openshift-operators/rhosdt
quay.v7.comptech-lab.com/openshift-operators/openshift4
quay.v7.comptech-lab.com/openshift-operators/openshift-service-mesh
quay.v7.comptech-lab.com/openshift-operators/openshift-service-mesh-tech-preview

GitOps render passed. Server-side dry-run passed for the new IDMS, the new namespaces, and the existing-namespace Service Mesh and Kiali Subscriptions. As in Batch 2, namespaced OperatorGroups and Subscriptions in namespaces created by the same render stream cannot be fully server-dry-run into not-yet-created namespaces; no live object was changed by that dry-run limitation.

The bootstrap clone on gf-ocp-bootstrap-01 was then fast-forwarded to 0f08a74.

Validation

Final validation:

Operator pods observed Running:

tempo-operator-controller
opentelemetry-operator-controller-manager
servicemesh-operator3
kiali-operator

Promotion decision

Keep the install.

The install is promoted because GitOps is synced, OLM is AtLatestKnown, all four CSVs are Succeeded, operator pods are Running, no excluded operands were created, and spoke cluster health stayed steady.

Rollback, if needed, is GitOps-first:

1. Remove the four Batch 3 operator directories from the spoke overlay.
2. Remove idms-tracing-mesh only if no other installed workload depends on those mirror mappings.
3. Sync and validate OLM/operator pruning.
4. Do not force-delete newly introduced CRDs or operator namespaces without a separate cleanup issue.

Operator queue decision:

OP-GF-OPERATORS-24: deferred by user decision; no active next operator batch is selected.

2026-05-21 App-platform batch deferral

Issue #440 records the user decision that the previously listed Batch 4 app platform operator wave is not needed now.

No live clusters, GitOps desired state, Subscriptions, operands, image mirror sets, routes, DNS records, or secrets were changed for this decision. Open Liberty and app-facing CloudNativePG remain catalog-visible and documented as deferred app-platform items, but future sessions must not present them as the next operator batch unless the user explicitly reintroduces application platform onboarding.