Domain and Network Map
The DNS plane, the HAProxy edge, the lab /24 boundary, and how a request travels from a browser to a backend service across the CompTech v6 lab.
This page is the network operator’s mental model for the CompTech v6 lab. It walks through the domains in use, the split between authoritative DNS and the lab recursor, the role of the HAProxy edge, the boundary between the lab /24 and public ingress, and the request-paths an end user actually traverses. Specific internal IP addresses are kept in the private connection-details/ runbooks; this page documents the topology and the rules that govern it.
Domain plane
The platform’s public DNS plane is sub.comptech-lab.com. Two derived wildcards do the heavy lifting:
| Domain | Backing record | Use |
|---|---|---|
sub.comptech-lab.com | apex + per-host A records | Authoritative zone — every lab service has a name here |
*.apps.sub.comptech-lab.com | wildcard A → HAProxy edge | Platform VM ingress + cluster ingress for hub-dc-v6 / spoke-dc-v6 |
*.mon.sub.comptech-lab.com | wildcard A → HAProxy edge | Parallel domain for the LGTM testing observability VM |
pdns.local | LAN-internal name | The PowerDNS VM itself |
*.apps.<cluster>.sub.comptech-lab.com | per-cluster wildcard | OpenShift route ingress; goes direct to cluster ingress VIP, not HAProxy |
api.<cluster>.sub.comptech-lab.com | per-cluster A record | OpenShift API server; goes direct to cluster API VIP, not HAProxy |
The wildcard certificate strategy mirrors the wildcard hostnames. There is one Let’s Encrypt wildcard for *.apps.sub.comptech-lab.com (used by HAProxy at 127.0.0.1:8443 for VM-side TLS termination), and one wildcard for *.mon.sub.comptech-lab.com (the monitoring track), both renewed via acme.sh --dns dns_pdns from the PowerDNS VM since the lab is internet-disconnected for inbound traffic and the DNS-01 challenge is the only practical issuance path.
The OpenShift clusters issue their own per-cluster ingress wildcards on *.apps.<cluster>.sub.comptech-lab.com. Those are independent of HAProxy and are terminated by the cluster’s ingress controller. HAProxy never terminates OpenShift route traffic.
Why two PowerDNS daemons on one VM
The pdns.local VM runs two daemons side-by-side, each with its own role:
- PowerDNS Authoritative holds the
sub.comptech-lab.comzone. SQLite-backed, single source of truth for every record in the lab. - PowerDNS Recursor is the default lab resolver. Every lab VM has it as its
/etc/resolv.confupstream. It forwards queries insidesub.comptech-lab.comto the local authoritative daemon on the same VM (over127.0.0.1) and recurses everything else to8.8.8.8and1.1.1.1.
The recursor refuses queries from outside the lab /16 (allow-from permits only loopback and the lab subnet). External clients cannot accidentally use the lab recursor; lab clients cannot accidentally bypass it. Public-facing authoritative queries arrive at the authoritative daemon’s separate public interface.
The reason for splitting authoritative and recursive is operational, not technical: the recursor handles “what is google.com” without ever touching the authoritative SQLite, and the authoritative handles “what is gitlab.apps.sub.comptech-lab.com” without forwarding upstream. Either daemon can be restarted, swapped, or replaced without touching the other.
DNS resolution flow
The Mermaid below traces how a lab VM resolves a name like nexus-mirror.apps.sub.comptech-lab.com:
- A lab VM asks the PowerDNS recursor for an
Arecord (e.g.,nexus-mirror.apps.sub.comptech-lab.com). - The recursor’s forward-zones rule for
sub.comptech-lab.comsends the query to the local authoritative server (127.0.0.1:53). - The authoritative server returns the HAProxy edge address.
- The recursor passes the answer back to the VM.
- The VM opens TLS
:443to the HAProxy edge.
For names outside sub.comptech-lab.com, the recursor uses forward-zones-recurse=.=8.8.8.8;1.1.1.1 and goes upstream. From the client’s perspective the recursor is the one resolver it ever needs to know about.
Network plane
The lab runs on a private 30.30.0.0/16 bridge (br30). All platform VMs live on this network; OpenShift nodes use a /24 slice for the cluster networks. IPv6 is explicitly disabled on the platform plane (DHCPv6 and Router Advertisements are administratively off-limits — ADR 0026 invariants).
Concrete addresses are kept in the private
connection-details/runbooks. This page intentionally documents the allocation pattern, not the per-host addresses.
Allocation pattern:
| Band | Used by | Reachable from |
|---|---|---|
Low /24 | Edge VMs (HAProxy, PowerDNS), platform VMs (GitLab, MinIO, Nexus, Vault, Jenkins, SigNoz, observability) | LAN + public via HAProxy |
Middle /24 | OpenShift cluster networks (machine, API VIP, ingress VIP, control-plane and worker nodes) | LAN-internal only; OpenShift routes via cluster ingress VIP |
High /24 | Physical workers (gold-1, gold-2, gpu-01), dynamic / future | LAN-internal |
The OpenShift cluster networks (pod, service, cluster) are private overlays defined by the cluster installer and not visible on the underlying lab br30 bridge except through the API and ingress VIPs.
Architecture: client → backend
Read the diagram as two concurrent flows:
DNS resolution (blue arrows at the top): the client asks the recursor; the recursor forwards sub.comptech-lab.com queries to the local authoritative daemon; the answer comes back via the recursor.
HTTPS request (orange / black arrows): the client opens TLS to HAProxy on :443. HAProxy uses SNI to identify the hostname, passes the TCP connection through to its own loopback on 127.0.0.1:8443, where the actual TLS handshake happens against the wildcard certificate. After decryption HAProxy can read the HTTP host header and route to the right backend VM.
OpenShift routes (red dashed) do not traverse HAProxy. The client resolves *.apps.<cluster>.sub.comptech-lab.com directly to the cluster ingress VIP and the cluster’s ingress controller terminates TLS with its own cert. This is the rule from feedback_haproxy_scope.md and is non-negotiable: HAProxy is for platform VM edge exposure only.
HAProxy: three roles in one VM
HAProxy on the edge VM does three jobs at once. They are coupled (one process, one config file) but operationally distinct:
- Public-internet edge for the small set of services that need internet ingress (DR public route, occasional vendor demo). Listens on public
:80and:443. This is the only TCP path from internet to lab; everything else is LAN-only. - Private-lab edge for the same set of hostnames from inside the
/16. Listens on the lab-network:80and:443. Same hostnames, same backends, different bind address. - SNI-passthrough → loopback TLS termination. All
vm-tls-be-routed traffic (the vast majority of platform-VM hostnames) is SNI-routed from:443to127.0.0.1:8443, where HAProxy decrypts with the wildcard cert and inspects the host header for backend selection. This pattern keeps the wildcard cert in one place — the loopback frontend — instead of duplicating it across every backend rule.
A handful of services use direct TCP passthrough rather than loopback re-decrypt:
- The (legacy, partially decommissioned) RKE2 cluster API on
:6443. - Kafka SNI passthrough for
bootstrap.kafka.apps.*andbroker-{0,1,2}.kafka.apps.*. - A WSO2 management split where the public API gateway and the admin console live on different upstream ports.
The /etc/haproxy/ directory keeps dated backups of every meaningful edit (the admin convention is haproxy.cfg.bak.YYYYMMDD-HHMM-<reason>). The current config is ~21 KB across ~40 frontend/backend/listen blocks; the original install was ~1.3 KB. Any new hostname requires the same shape: ACL entry, use_backend rule, backend block, dated backup before edit.
The lab /24 vs the public ingress boundary
Two distinct “edges” matter:
- Lab
/24edge is the boundary of the private RFC1918 network. Everything reachable on30.30.0.0/16is “inside the lab.” Reaching it from outside requires either VPN access, a public-bound HAProxy frontend, or direct cluster ingress for*.apps.<cluster>.sub.comptech-lab.com. - Public ingress is the subset of services that have an internet-routable hostname (the
*.apps.sub.comptech-lab.comwildcard, fronted by HAProxy’s public bind). Most platform VMs are reachable here. OpenShift routes are not — they’re cluster-direct, not HAProxy-fronted.
A request originating from inside the lab to gitlab.apps.sub.comptech-lab.com resolves through the recursor to HAProxy’s LAN-network bind address and terminates there. The same request from outside the lab resolves (via public DNS) to HAProxy’s public bind address and terminates there. The two are different addresses on the same HAProxy VM, with identical backend rules.
For OpenShift routes:
| Origin | Resolves to | Terminates at |
|---|---|---|
| Inside lab | cluster ingress VIP (LAN address) | OpenShift ingress controller |
| Outside lab | cluster ingress VIP (public address) | OpenShift ingress controller |
There is no HAProxy in either path. The OpenShift cluster owns its ingress.
Hostname inventory — what lives where
The HAProxy host-header ACLs (acl is_vm_host, acl is_wso2_host) enumerate the platform-VM hostnames. Today’s inventory:
Core platform-VM hostnames (all *.apps.sub.comptech-lab.com): gitlab, minio, minio-console, haproxy, vault-rke2, auth, nexus, nexus-docker, nexus-mirror, mirror-registry, docker-group, app-registry, defectdojo, jenkins, signoz, trivy, monitoring, grafana.
WSO2 split (legacy): is, apim, publisher, devportal, admin, gateway.
Kafka SNI: bootstrap.kafka, broker-{0,1,2}.kafka.
OpenShift cluster ingress (not via HAProxy): *.apps.hub-dc-v6.sub.comptech-lab.com, *.apps.spoke-dc-v6.sub.comptech-lab.com.
OpenShift cluster APIs (not via HAProxy): api.hub-dc-v6.sub.comptech-lab.com:6443, api.spoke-dc-v6.sub.comptech-lab.com:6443.
Adding a new edge-exposed service
The mechanical steps for a new platform-VM hostname (e.g., a new tool VM tooling.apps.sub.comptech-lab.com):
- DNS: add A record on the authoritative PowerDNS for
tooling.apps.sub.comptech-lab.com→ HAProxy edge address. Usepdnsutil add-recordor the HTTP API; changes are immediate, no zone reload needed. - HAProxy: backup the config first (
cp haproxy.cfg haproxy.cfg.bak.$(date -u +%Y%m%d-%H%M).<reason>). Add the hostname to theis_vm_hostACL list, add theuse_backendrule on thevm-tlsfrontend, and add the backend block pointing at the tool VM’s LAN address + port. Reload HAProxy. - Cert: the existing
wildcard-apps.pemalready covers anything under*.apps.sub.comptech-lab.com. No new cert needed. - Validate:
dig @<recursor> tooling.apps.sub.comptech-lab.com A +shortreturns the HAProxy edge;curl -sSI https://tooling.apps.sub.comptech-lab.com/returns the backend’s response.
For a *.mon.sub.comptech-lab.com hostname the pattern is slightly different — five rule additions across SNI rule, vm-tls host-header → backend, public-apps-http redirect ACL, vm-tls deny-unless ACL, plus the new backend if needed — because the monitoring wildcard is loaded on a separate cert and HAProxy needs explicit SNI rules to pick the right certificate at the loopback termination.
Common DNS/edge failure modes
| Symptom | Likely cause | Where to look |
|---|---|---|
NXDOMAIN from inside lab | record missing on authoritative | sudo pdnsutil show-zone sub.comptech-lab.com on pdns.local |
NXDOMAIN only from a specific VM | VM points at a different resolver | cat /etc/resolv.conf on that VM; should list the lab recursor |
| TLS handshake error | hostname doesn’t match cert SAN | wildcard covers *.apps.sub.comptech-lab.com — confirm hostname uses that exact wildcard |
| HTTP 503 from edge | backend down or backend cert/connection failure | HAProxy stats page; check the backend VM service |
| Wrong backend served for hostname | host-header rule missing in HAProxy | re-check the acl and use_backend block for the hostname |
| Cluster API unreachable | API VIP routing problem | api.<cluster>.sub.comptech-lab.com:6443 resolves direct, not via HAProxy — investigate cluster networking, not HAProxy |
| OpenShift route returns wrong cert | route hostname doesn’t match cluster ingress wildcard | the cluster wildcard is *.apps.<cluster>.sub.comptech-lab.com — different cert than the HAProxy wildcard |
Backup, recovery, and HA posture
The lab runs a single PowerDNS VM and a single HAProxy VM today. Both are recognized single-VM dependencies; HA is a future ADR (currently out of scope per the workspace scope). Backup posture:
- PowerDNS: SQLite zone file at
/var/lib/powerdns/pdns.sqlite3; dated config backups in/etc/powerdns/*.bak.YYYYMMDD-HHMM-<reason>. Zone export available viapdnsutil show-zone sub.comptech-lab.com. No replication; ifpdns.localis lost, every record needs reconstruction from the SQLite snapshot or the operator handoff doc. - HAProxy: plaintext config at
/etc/haproxy/haproxy.cfg; dated backups in the same directory. Restore iscp haproxy.cfg.bak.<latest> haproxy.cfg && systemctl restart haproxy.
If either VM is lost the lab degrades hard. The operating model treats both as critical-dependency hardware that gets a dated config backup before every meaningful edit.
References
connection-details/platform-admin-handoff.md(private) — network details table, cluster API/ingress VIPsconnection-details/nexus.md(private) — three-endpoint DNS recordsadr/0019-nexus-only-image-supply-chain.md— runtime registry hostname allowlistadr/0026-ipv6-for-ovn-k-amendment.md— IPv6 invariants