Installation Manual - 24 Spoke ACM Import Pull GitOps
How to import spoke-dc-v7 into hub-dc-v7 ACM and enable the pull-model OpenShift GitOps flow.
This chapter imports spoke-dc-v7 into hub-dc-v7 ACM and enables the
pull-model GitOps pattern used for managed clusters.
Prerequisites
hub-dc-v7is healthy and ACM/MCE is running.spoke-dc-v7is healthy.- LSO/ODF is already installed on the spoke.
- The greenfield operational GitOps repo exists in GitLab:
http://gitlab.v7.comptech-lab.com/platform/openshift/openshift-gitops.git. - You can access both kubeconfigs from
gf-ocp-bootstrap-01.
Set kubeconfigs:
export HUB_KUBECONFIG=/home/ze/ocp-greenfield-deployment/artifacts/openshift/hub-dc-v7/auth/kubeconfig
export SPOKE_KUBECONFIG=/home/ze/ocp-greenfield-deployment/artifacts/openshift/spoke-dc-v7/auth/kubeconfig
Validate both clusters first:
oc --kubeconfig "$HUB_KUBECONFIG" get clusterversion
oc --kubeconfig "$HUB_KUBECONFIG" get nodes
oc --kubeconfig "$HUB_KUBECONFIG" get multiclusterhub -n open-cluster-management
oc --kubeconfig "$SPOKE_KUBECONFIG" get clusterversion
oc --kubeconfig "$SPOKE_KUBECONFIG" get nodes
Import The Spoke
Create the hub-side import resources:
oc --kubeconfig "$HUB_KUBECONFIG" create namespace spoke-dc-v7
oc --kubeconfig "$HUB_KUBECONFIG" apply -f managedcluster-spoke-dc-v7.yaml
oc --kubeconfig "$HUB_KUBECONFIG" apply -f klusterletaddonconfig-spoke-dc-v7.yaml
Important: if the import namespace is created at the same time as the
ManagedCluster, ACM can terminate and recreate that namespace while generating
bootstrap token content. If token generation fails with a namespace terminating
or not-found error, recreate the namespace after the ManagedCluster exists,
then recreate the KlusterletAddonConfig.
Wait for the import secret:
oc --kubeconfig "$HUB_KUBECONFIG" get secret -n spoke-dc-v7 spoke-dc-v7-import
Apply only the generated import manifests to the spoke. Do not print the secret contents:
workdir="$(mktemp -d)"
oc --kubeconfig "$HUB_KUBECONFIG" get secret -n spoke-dc-v7 spoke-dc-v7-import \
-o jsonpath='{.data.crds\.yaml}' | base64 -d > "$workdir/crds.yaml"
oc --kubeconfig "$HUB_KUBECONFIG" get secret -n spoke-dc-v7 spoke-dc-v7-import \
-o jsonpath='{.data.import\.yaml}' | base64 -d > "$workdir/import.yaml"
oc --kubeconfig "$SPOKE_KUBECONFIG" apply -f "$workdir/crds.yaml"
oc --kubeconfig "$SPOKE_KUBECONFIG" apply -f "$workdir/import.yaml"
rm -rf "$workdir"
Validate:
oc --kubeconfig "$HUB_KUBECONFIG" get managedcluster spoke-dc-v7
oc --kubeconfig "$HUB_KUBECONFIG" get managedclusteraddon -n spoke-dc-v7
Expected result:
HUB ACCEPTED=true;JOINED=True;AVAILABLE=True;- all expected add-ons report
Available=True.
Codify Pull GitOps
In the operational GitOps repo, add a hub fleet-registration slice:
clusters/hub-dc-v7/platform/fleet-registration/
It must include:
ManagedCluster/spoke-dc-v7;KlusterletAddonConfig/spoke-dc-v7;ManagedClusterSetBinding/defaultinopenshift-gitops;Placement/gitops-managed;GitOpsCluster/gitops-managed;ConfigMap/acm-placement;Namespace/spoke-dc-v7.
Also add a hub GitOps control ApplicationSet:
clusters/hub-dc-v7/gitops-control/applicationset-spoke-cluster-config-pull.yaml
Use the greenfield GitLab repo URL, not the previous environment URL:
repoURL: http://gitlab.v7.comptech-lab.com/platform/openshift/openshift-gitops.git
path: "clusters/{{name}}"
Create the spoke path:
clusters/spoke-dc-v7/
For the initial takeover, codify the already-installed storage layer:
- Local Storage Operator subscription;
- ODF subscription;
- exact-path
LocalVolumeresources; StorageCluster/ocs-storagecluster;- default
ocs-storagecluster-ceph-rbdStorageClass annotation.
Render before pushing:
oc kustomize clusters/hub-dc-v7
oc kustomize clusters/spoke-dc-v7
git diff --check
Commit and push:
git add clusters/hub-dc-v7 clusters/spoke-dc-v7
git commit -m "Register spoke-dc-v7 with ACM pull GitOps"
git push origin main
Refresh the hub bootstrap app:
oc --kubeconfig "$HUB_KUBECONFIG" annotate applications.argoproj.io \
-n openshift-gitops hub-dc-v7-bootstrap \
argocd.argoproj.io/refresh=hard --overwrite
Seed Private Repo Access
The generated spoke Argo instance runs on the managed cluster. It needs a repository credential to read the private GitLab repository.
At this stage of the install, spoke-dc-v7 does not yet have External Secrets
Operator or Vault Kubernetes auth. Do not try to put this repository Secret in
the GitOps repo; Argo cannot read the GitOps repo until this credential exists.
Use the bootstrap script from the greenfield deployment repo instead:
/home/ze/ocp-greenfield-deployment/scripts/services/bootstrap/seed-managed-gitops-repo-credential.sh \
--cluster spoke-dc-v7 \
--kubeconfig "$SPOKE_KUBECONFIG"
The script reads the repository credential from Vault and creates only the Argo CD repository Secret in the managed cluster:
Vault path:
secret/greenfield/openshift/spoke-dc-v7/gitops/repositories/openshift-gitops
Target Secret:
openshift-gitops/openshift-gitops-gitlab-repo
Expected Vault keys:
url;username;password.
Required environment:
export VAULT_ADDR=https://gf-ocp-vault-02.v7.comptech-lab.com:8200
export VAULT_TOKEN=<token with read access to the path>
If the bootstrap host does not yet trust the greenfield Vault CA, use
VAULT_SKIP_VERIFY=true only for this bootstrap step and fix CA trust later.
The script does not print the password or token. It also writes the Kubernetes
Secret with password from a temporary file, so the password is not passed as
a command-line argument.
Validate the credential by refreshing the spoke app:
oc --kubeconfig "$SPOKE_KUBECONFIG" annotate applications.argoproj.io \
-n openshift-gitops spoke-dc-v7-cluster-config \
argocd.argoproj.io/refresh=hard --overwrite
oc --kubeconfig "$SPOKE_KUBECONFIG" get applications.argoproj.io \
-n openshift-gitops spoke-dc-v7-cluster-config
oc --kubeconfig "$SPOKE_KUBECONFIG" get secret \
-n openshift-gitops openshift-gitops-gitlab-repo \
-o jsonpath='secretType={.metadata.labels.argocd\.argoproj\.io/secret-type}{"\n"}'
Expected result:
spoke-dc-v7-cluster-configisSynced/Healthy;- the repository Secret label is
argocd.argoproj.io/secret-type=repository.
Final Validation
Hub:
oc --kubeconfig "$HUB_KUBECONFIG" get applications.argoproj.io -n openshift-gitops hub-dc-v7-bootstrap
oc --kubeconfig "$HUB_KUBECONFIG" get managedcluster spoke-dc-v7
oc --kubeconfig "$HUB_KUBECONFIG" get managedclusteraddon -n spoke-dc-v7
oc --kubeconfig "$HUB_KUBECONFIG" get placement -n openshift-gitops gitops-managed
oc --kubeconfig "$HUB_KUBECONFIG" get gitopscluster -n openshift-gitops gitops-managed
oc --kubeconfig "$HUB_KUBECONFIG" get applicationset -n openshift-gitops spoke-cluster-config-pull
oc --kubeconfig "$HUB_KUBECONFIG" get applications.argoproj.io -n openshift-gitops spoke-dc-v7-cluster-config
Spoke:
oc --kubeconfig "$SPOKE_KUBECONFIG" get applications.argoproj.io -n openshift-gitops spoke-dc-v7-cluster-config
oc --kubeconfig "$SPOKE_KUBECONFIG" get storagecluster -n openshift-storage ocs-storagecluster
oc --kubeconfig "$SPOKE_KUBECONFIG" get cephcluster -n openshift-storage ocs-storagecluster-cephcluster
oc --kubeconfig "$SPOKE_KUBECONFIG" get sc ocs-storagecluster-ceph-rbd
Expected final state:
- hub bootstrap app:
Synced/Healthy; ManagedCluster/spoke-dc-v7: joined and available;gitops-addon: available;- hub-generated spoke app:
Synced/Healthy; - spoke-side
spoke-dc-v7-cluster-config:Synced/Healthy; StorageCluster/ocs-storagecluster:Ready;- Ceph:
HEALTH_OK; ocs-storagecluster-ceph-rbd: default StorageClass.