GitOps Operating Model — Architecture

The ACM + OpenShift GitOps Basic pull model in detail: hub Argo + spoke Argo + ApplicationSet + ManifestWork, what initiates what, and why pull beats hub-render at fleet scale.

The fleet’s operating model is ACM + OpenShift GitOps Basic pull (per ADR 0018). Both clusters run an Argo CD instance that reconciles its own desired state from the same GitLab repository, with the hub Argo coordinating placement and ApplicationSet generation but never directly applying spoke manifests. The spoke is its own reconciler. This page is the architecture in one diagram and one set of definitions.

The architecture

The defining property: every dashed-green arrow is initiated from the inside out. The klusterlet on the spoke calls the hub. The spoke Argo CD calls GitLab. The hub Argo CD calls GitLab. Nothing on the hub initiates a connection to the spoke API server. This is what makes the pattern survive NAT, firewalls, transient WAN, and (in larger fleets) air-gapped sites between sync windows.

The two Argo CD instances

InstanceWhereInstallPrivilegeReconciles
Hub Argo CDhub-dc-v6 openshift-gitops namespaceOLM CSV openshift-gitops-operator.v1.20.3cluster-admin ClusterRoleBindingclusters/hub-dc-v6/ (against kubernetes.default.svc) plus the per-spoke Application objects (which it does not apply)
Spoke Argo CDspoke-dc-v6 openshift-gitops namespaceDeployment installed by ACM gitops-addonLeast-privilege ClusterRole argocd-platform-extensionsclusters/spoke-dc-v6/ (against kubernetes.default.svc)

The hub uses a bound cluster-admin because it has to install ACM, MCE, all platform operators, RHACS Central, and write cluster-scoped resources — a least-privilege ClusterRole would be a fast-moving target as new operators land. The compensating controls are: the hub Argo’s AppProject/hub-dc-v6-platform sourceRepos allowlist restricts it to the LAN GitLab URL only; CODEOWNERS in GitLab restricts who can merge; argocd-platform-extensions doesn’t exist on the hub because it isn’t needed.

The spoke uses a least-privilege ClusterRole because it is exposed by ACM’s pull pattern. The addon-installed Argo’s service account acm-openshift-gitops-argocd-application-controller has get/list/watch on everything plus full verbs on a fixed allowlist of API groups (see §5.5 spoke extension RBAC).

The Application objects, in three roles

The same Argo Application CR plays three different roles depending on where it lives and what labels it carries:

Role 1 — hub bootstrap Application

hub-dc-v6-bootstrap lives in the hub Argo’s openshift-gitops namespace and reconciles clusters/hub-dc-v6/ against kubernetes.default.svc (i.e., the hub itself).

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: hub-dc-v6-bootstrap
  namespace: openshift-gitops
spec:
  project: hub-dc-v6-platform
  source:
    repoURL: http://<gitlab-vm>/comptech-platform/openshift-ops/openshift-platform-gitops.git
    targetRevision: main
    path: clusters/hub-dc-v6
  destination:
    server: https://kubernetes.default.svc
    namespace: openshift-gitops
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
      - ServerSideApply=true

This is the only object an operator has to oc apply by hand (after the OpenShift GitOps operator is installed). From there everything else flows through GitOps.

Role 2 — ApplicationSet-generated “Application as ManifestWork wrapper”

The hub Argo’s spoke-cluster-config-pull ApplicationSet generates one Application per spoke. Critical features of the generated Application:

  • metadata.labels.apps.open-cluster-management.io/pull-to-ocm-managed-cluster: "true" — tells ACM to wrap it as ManifestWork.
  • metadata.annotations.argocd.argoproj.io/skip-reconcile: "true" — tells the hub Argo CD to not reconcile it. The hub Argo is the owner-of-record only; ACM ships it to the spoke.
  • metadata.annotations.apps.open-cluster-management.io/ocm-managed-cluster: "{{name}}" — tells ACM which spoke to target.
  • spec.project: default — lands on the spoke, where hub-dc-v6-platform AppProject doesn’t exist; default does.
  • spec.destination.server: https://kubernetes.default.svc — points at the spoke’s local API once it lands there.

Role 3 — spoke-side Application

When the ManifestWork lands on the spoke, the wrapped Application is materialized in the spoke’s openshift-gitops namespace. The spoke Argo CD finds it and reconciles it normally — pulling clusters/spoke-dc-v6/ from GitLab and applying to the local API.

Net effect: from the spoke Argo’s perspective, it has a single Application called spoke-dc-v6-cluster-config that reconciles clusters/spoke-dc-v6/. From the hub’s perspective, the same Application was generated by an ApplicationSet and shipped via ManifestWork.

What initiates what

ConnectionInitiated byDirection
klusterlet → hub registration controllerklusterlet (spoke)outbound from spoke
klusterlet work agent → ACM ManifestWork pullklusterlet (spoke)outbound from spoke
Hub Argo → GitLab clusters/hub-dc-v6/hub Argooutbound from hub
Spoke Argo → GitLab clusters/spoke-dc-v6/spoke Argooutbound from spoke
Spoke Argo → spoke API serverspoke Argolocal (in-cluster)
Hub Argo → hub API serverhub Argolocal (in-cluster)
Hub Argo → spoke API servernevern/a

The last row is the whole point. Without it, the hub doesn’t need spoke kubeconfigs, doesn’t need to reach the spoke’s API, doesn’t need a hole punched through a NAT for spoke API access.

Sync-wave ordering, in summary

OpenShift GitOps respects argocd.argoproj.io/sync-wave annotations. The convention used across platform-gitops:

WaveRole
0RBAC scaffolding (ClusterRoles, ClusterRoleBindings, hub Argo cluster-admin binding, spoke argocd-platform-extensions, the AppProject manager Role for the bootstrap namespace)
5Namespace, OperatorGroup
10Subscription (operator install)
20Operator CR (MultiClusterHub, ClusterCatalog, IDMS/ITMS, StorageCluster, LokiStack, …)
30First-class operand objects (ManagedCluster, KlusterletAddonConfig, ManagedClusterSetBinding)
40Per-spoke addon config
50Placement
60GitOpsCluster
70ApplicationSet

Details and the full per-resource table in §5.3 sync-wave conventions.

What this isn’t

This model is not the Argo CD Agent (now Argo CD ApplicationSet pull controller in newer versions). Argo CD Agent installs an “agent” Argo on the spoke that talks to a “principal” Argo on the hub via a dedicated bidirectional channel, and disables the hub Argo’s normal application controller in the process. ADR 0018 explicitly rejects this for the v6 baseline:

Do not use Argo CD Agent for this baseline. It may be reconsidered later only through a separate accepted ADR and after current Red Hat support, maturity, and operational load implications are reviewed.

The two distinctions:

ACM + OpenShift GitOps Basic pull (this fleet)Argo CD Agent
Spoke Argo installACM gitops-addon DeploymentArgo CD Agent install
ManifestWork dispatchYes — ACM’s app-lifecycle pillar wraps AppsNo — Agent has its own dispatch channel
Hub Argo behaviorNormal — owns hub-side AppSet, doesn’t reconcile spoke AppsDemoted to “principal”, doesn’t run application-controller for spoke targets
Cluster registrationACM ManagedClusterArgo CD Agent registration
Maturity in OpenShiftStable since OCP 4.16Newer, less Red Hat-tested

We chose Basic pull because it leans on ACM components that are already needed for cluster lifecycle, governance, and the multicluster console.

References

  • adr/0003-gitops-basic-pull-model.md — pre-v6 origin of the pull model.
  • adr/0018-acm-openshift-gitops-pull-model-v6.md — current decision for the v6 fleet.
  • connection-details/openshift-gitops-hub-dc-v6.md — hub Argo CD instance + bootstrap Application.
  • connection-details/platform-admin-handoff.md — operating workflow summary.
  • platform-gitops clusters/hub-dc-v6/gitops-control/ — the hub-side glue (AppProject, ApplicationSet, cluster-admin binding).

Last reviewed: 2026-05-11