Installation Manual - 07 Quay registry

How the gf-ocp Quay registry VM is created, connected to MinIO object storage, routed through HAProxy, and validated.

Quay is the greenfield platform registry foundation. It is installed after MinIO, PowerDNS, HAProxy, GitLab, Vault, and NetBox because it depends on object storage, DNS, edge TLS, secret custody, and source-of-truth inventory.

Target State

PowerDNS records:

quay.v7.comptech-lab.com         A 59.153.29.102
quay-private.v7.comptech-lab.com A 30.30.200.40

Creation Flow

The operator entry point is the greenfield repository script wrapper:

./scripts/gfctl.sh prepare-cloud-init --execute gf-ocp-quay-01
./scripts/gfctl.sh cloud-init-iso --execute gf-ocp-quay-01
cp scripts/vms/gf-ocp-quay-01.env.example scripts/vms/gf-ocp-quay-01.env
./scripts/gfctl.sh create-vm --execute scripts/vms/gf-ocp-quay-01.env

Before building the seed ISO, replace placeholder SSH public keys in:

artifacts/cloud-init/gf-ocp-quay-01/user-data.yaml

The VM is private-only. HAProxy is the only public entry point.

Runtime Install

The install script is:

/usr/local/sbin/install-quay-standalone.sh

It installs and enables:

• local PostgreSQL 16;
• local Redis 7 with authentication;
• Podman;
• Project Quay container quay.io/projectquay/quay:3.17.1;
• quay.service;
• Quay standalone config at /etc/quay/config/config.yaml.

The runtime env file is root-only:

/etc/quay/quay.env

It is generated from Vault-held values and must never be committed or printed.

Object Storage

Quay uses RadosGWStorage because MinIO is S3-compatible. The registry talks to MinIO over the private service network:

http://30.30.200.1:9000

Client-facing TLS remains on HAProxy. Quay is configured with EXTERNAL_TLS_TERMINATION: true, PREFERRED_URL_SCHEME: https, and SESSION_COOKIE_SECURE: true.

HAProxy Route

HAProxy redirects HTTP to HTTPS for quay.v7.comptech-lab.com and forwards the HTTPS route to:

30.30.200.40:8080

The public certificate is the shared wildcard certificate for *.v7.comptech-lab.com.

Validation

Run local validation on the VM:

ssh ze@30.30.200.40 'sudo /usr/local/sbin/validate-quay-standalone.sh /etc/quay/quay.env'

Run edge validation from an operator host:

curl -fsS https://quay.v7.comptech-lab.com/api/v1/discovery >/dev/null

The first authenticated push/pull smoke test used the initialized zahid administrator account and pushed:

quay.v7.comptech-lab.com/zahid/smoke:op-gf-quay1

Backup And Restore

Quay backup artifacts are encrypted before upload to MinIO:

quay-backups/quay/YYYYMMDDTHHMMSSZ/quay-YYYYMMDDTHHMMSSZ.tar.gz.enc
quay-backups/quay/YYYYMMDDTHHMMSSZ/quay-YYYYMMDDTHHMMSSZ.tar.gz.enc.sha256

The backup includes the Quay PostgreSQL database dump, Quay config, systemd unit, and backup metadata. The backup credential and archive passphrase are stored at:

secret/greenfield/object-storage/minio/users/quay-backup

Installed files on gf-ocp-quay-01:

/etc/quay/quay-backup.env
/usr/local/sbin/quay-backup-to-minio.sh
/usr/local/sbin/quay-restore-validate.sh
quay-backup.service
quay-backup.timer

Validation:

ssh ze@30.30.200.40 'sudo /usr/local/sbin/quay-backup-to-minio.sh'
ssh ze@30.30.200.40 'sudo /usr/local/sbin/quay-restore-validate.sh'
ssh ze@30.30.200.40 'sudo systemctl list-timers quay-backup.timer --no-pager'

The quay-backups bucket has versioning enabled and a 90-day lifecycle policy for current and noncurrent backup objects.

Repository And Robot Model

The source-of-truth bootstrap model is:

data/quay/bootstrap-model.json

Namespaces:

• openshift-release
• openshift-operators
• platform
• golden-images
• tenants

Initial repositories:

• openshift-release/release-images
• openshift-release/openshift/release-images
• openshift-release/release-metadata
• openshift-operators/redhat-operator-index
• openshift-operators/certified-operator-index
• platform/ci-tools
• platform/smoke
• golden-images/ubi9

Initial organization robots:

• openshift-release+ocp_mirror
• openshift-operators+ocp_mirror
• platform+ci_push
• platform+read_only
• golden-images+read_only

Apply the model during bootstrap from the Quay VM:

sudo /usr/local/sbin/seed-quay-internal-model.sh \
  /etc/quay/bootstrap/bootstrap-model.json \
  /root/quay-robot-tokens.json

The internal seeder is used during standalone bootstrap because the initial Quay user-initialize token does not provide the full organization, repository, robot, and permission API scope required for this model. After seeding, move robot credentials directly into Vault and remove any temporary local copies.

Validated smoke tags:

quay.v7.comptech-lab.com/openshift-release/release-metadata:op-gf-quay2-smoke
quay.v7.comptech-lab.com/openshift-operators/certified-operator-index:op-gf-quay2-smoke
quay.v7.comptech-lab.com/platform/smoke:op-gf-quay2-smoke

Future oc-mirror work should use the release and operator mirror robots and store generated pull-secret material in Vault, not Git.

The release and operator mirror robots are members of namespace-local mirror-creators teams with the Quay creator role. This is required because oc-mirror creates destination repositories from source image paths.

Quay must be configured with extended repository names enabled:

FEATURE_EXTENDED_REPOSITORY_NAMES: true

This is required for oc-mirror v2. The release mirror writes nested paths such as openshift-release/openshift/release-images, and operator mirrors include deep paths such as openshift-operators/appcafe/open-liberty/samples/getting-started.

Before the production operator mirror, pre-create repositories from the oc-mirror dry-run mapping and grant the mirror robot write access:

scp scripts/services/quay/seed-oc-mirror-repositories-from-mapping.sh \
  artifacts/oc-mirror/production-operators/working-dir/dry-run/mapping.txt \
  ze@30.30.200.40:/tmp/
ssh ze@30.30.200.40 \
  'sudo install -m 0755 /tmp/seed-oc-mirror-repositories-from-mapping.sh /usr/local/sbin/seed-oc-mirror-repositories-from-mapping.sh &&
   sudo /usr/local/sbin/seed-oc-mirror-repositories-from-mapping.sh \
     --mapping /tmp/mapping.txt \
     --organization openshift-operators \
     --robot openshift-operators+ocp_mirror'

The May 15, 2026 production run seeded 501 operator destination repositories, then completed 193 / 193 release images, 582 / 582 operator images, and 4 / 4 additional images.

Lessons Captured

• Upstream Project Quay image tags use the numeric form, for example 3.17.1, not v3.17.1.
• URL-encode the PostgreSQL password inside DB_URI.
• Quote generated secret values in YAML.
• Keep internal S3 traffic on the private MinIO endpoint instead of sending it through the public HAProxy route.
• Validate the scoped MinIO user with mc before starting Quay.
• Use unique organization email aliases in the Quay bootstrap model.
• Enable extended repository names before running oc-mirror.
• Pre-seed operator repositories from the oc-mirror dry-run mapping before the long production operator mirror.