PowerDNS architecture

The dual-homed single-VM PowerDNS deployment: authoritative + recursor on one host, what each daemon does, and how queries flow from lab VMs to the lab zone and onward to the public internet.

DNS in this lab is one VM, two daemons, one SQLite file. That sentence is the whole architecture; the rest of this page expands on what each piece does and why. The dual-homed single-host shape was inherited from the previous lab generation and intentionally kept: it is the smallest viable design that still gives lab clients a real recursor without coupling them to a public resolver, and still answers authoritatively for the sub.comptech-lab.com zone.

This page covers the architecture. The other DNS subsection pages cover the zone contents, the forwarder configuration, and the failure modes / recovery procedures.

The host

The public NIC carries an externally reachable address. The private NIC carries two addresses on the lab /16: one used by the authoritative daemon (the historical auth address for the lab zone) and one used by the recursor (the address every lab VM points at for resolution). Two daemons, three IPs, one VM.

Diagram

The defining shape:

• Lab VMs and the workstation point at the recursor address — not the authoritative address.
• The recursor forwards anything ending in sub.comptech-lab.com to the authoritative daemon on the same VM (over loopback), and recurses everything else to public resolvers (8.8.8.8, 1.1.1.1).
• Public-internet DNS users (anyone resolving the lab’s external zone) hit the authoritative daemon directly via the public NIC. They never use the recursor.
• The SQLite file is read by both auth daemons (same process, two binds) and is the entire source of truth for the zone.

Two daemons, side by side

Both daemons are active on the same host (systemctl status pdns pdns-recursor returns both as active (running)).

This split matters because clients are not supposed to send recursive queries to an authoritative server. PowerDNS Authoritative does not do recursion at all — if you ask it for google.com, it returns REFUSED. The recursor is the front door for recursive queries; the authoritative is the source of truth for sub.comptech-lab.com. The same VM runs both because the lab is small enough not to need separation, but they remain functionally distinct.

The authoritative side

The authoritative config (/etc/powerdns/pdns.conf) key fields:

launch=gsqlite3
gsqlite3-database=/var/lib/powerdns/pdns.sqlite3
gsqlite3-pragma-synchronous=0

local-address=127.0.0.1,<public-auth-ip>,<private-auth-ip>
local-port=53

webserver=yes
webserver-address=127.0.0.1
webserver-port=8081
api=yes
api-key=<API_KEY>  # NOT printed in this file

Key choices:

• launch=gsqlite3 — the simplest backend that supports the PowerDNS HTTP API. A SQLite file is overkill simple; perfectly fine for a few hundred records and a single operator.
• gsqlite3-pragma-synchronous=0 — disables fsync on every write. Trades durability for write performance. Acceptable for a lab; for a real shared zone you’d reverse this.
• webserver=yes + api=yes on 127.0.0.1:8081 — the API key is in pdns.conf and is loopback-only. Record changes are made via this API, never by editing the SQLite directly. The API binding is intentionally not exposed on 0.0.0.0.
• No AXFR, no DNS NOTIFY, no replication. This is a single-node design. If high-availability matters, that’s a future ADR.

The authoritative daemon does NOT recurse. Querying it for a name outside the zone returns REFUSED — that is correct, and is the design.

The recursor side

The recursor config (/etc/powerdns/recursor.conf plus /etc/powerdns/recursor.d/10-lab-forwarder.conf):

local-address=127.0.0.1,<private-recursor-ip>
local-port=53

allow-from=127.0.0.0/8,<lab-/16>

# Lab subdomain → local authoritative (loopback)
forward-zones=sub.comptech-lab.com=127.0.0.1:53

# Everything else → public resolvers
forward-zones-recurse=.=8.8.8.8;1.1.1.1

Key choices:

• allow-from — only loopback and the lab /16 are allowed to query. From outside the lab the recursor refuses. This is what keeps the recursor from being an open resolver on the internet.
• forward-zones=sub.comptech-lab.com=127.0.0.1:53 — when a lab VM asks for vault.sub.comptech-lab.com, the recursor forwards the query to the local authoritative daemon over loopback rather than trying to recurse on the public root. This avoids leaking lab names through public resolvers and avoids depending on whether the public delegation for the zone is up to date.
• forward-zones-recurse=.=8.8.8.8;1.1.1.1 — everything outside the lab zone is forwarded with the recursive bit to Google + Cloudflare. The recursor does not perform full iterative resolution from the root; it forwards. This is fine for a lab.

The recursor has no HTTP API exposed to clients.

What clients see

Every Ubuntu cloud-init VM and every OpenShift node points at the recursor address as the only resolver, with ~sub.comptech-lab.com as a search/route domain. Internal-only resolver behavior is therefore:

• dig @<lab-recursor> vault.sub.comptech-lab.com → answered by the local authoritative daemon (the recursor forwards over loopback).
• dig @<lab-recursor> google.com → answered by 8.8.8.8 via the recursor’s forwarder.
• dig @<lab-auth> google.com → REFUSED (auth doesn’t recurse). This is normal.
• From outside the lab /16, dig @<lab-recursor> ... → no response (allow-from denies).

This matters when debugging: “DNS works on this VM but not from another VM” usually means a VM was misconfigured to point at the authoritative address rather than the recursor address. Symptom: external names fail with REFUSED, lab names work. Fix: change the resolver to the recursor address.

SQLite zone storage

The zone file isn’t a file — it’s rows in pdns.sqlite3. To enumerate or edit:

# enumerate zones
sudo pdnsutil list-all-zones

# show all records for the lab zone
sudo pdnsutil show-zone sub.comptech-lab.com

# add an A record (e.g., a new platform VM)
sudo pdnsutil add-record sub.comptech-lab.com new-vm A 3600 <ip>

# replace an entire RRset
sudo pdnsutil replace-rrset sub.comptech-lab.com new-vm A 3600 <ip>

# delete an RRset
sudo pdnsutil delete-rrset sub.comptech-lab.com new-vm A

pdnsutil writes the SQLite directly and bumps the SOA serial automatically. Because the SQLite is the live store, changes are immediate — no zone reload, no rsync.

The HTTP API on 127.0.0.1:8081 is the alternative. The API is what GitHub Actions / acme.sh use for ACME DNS-01 challenges (e.g., the wildcard-mon.pem LE cert for *.mon.sub.comptech-lab.com is issued via acme.sh --dns dns_pdns against this API).

Why this single-VM design is acceptable

• Lab scale. A few hundred records, a single operator, no SLO. PowerDNS on SQLite handles this with zero tuning.
• One source of truth. The SQLite file is the entire authoritative state. Backups are trivial (copy the file). Audit is trivial (pdnsutil show-zone).
• No public delegation drama. The recursor’s forward-zones=sub.comptech-lab.com=127.0.0.1:53 line means lab clients don’t need the public delegation for sub.comptech-lab.com to be working — they hit the local authoritative directly.
• The recursor is intentional. Clients ask the recursor, never the authoritative. That gives clients a real resolver (caching, NXDOMAIN, TTLs) without coupling them to a specific public resolver.

What this design is not

• Not HA. Lose the VM, lose lab DNS. Recovery is a SQLite restore from backup; mitigation for a longer outage is on the failure-modes page.
• Not a public secondary. No AXFR allowed, no NOTIFY targets. The public delegation for sub.comptech-lab.com resolves against the authoritative bind on the public NIC, but there is no secondary nameserver.
• Not a DNSSEC-validating resolver. DNSSEC=no on host stubs. Lab traffic is internal; the trade-off is fewer moving parts.

References

• opp-full-plat/plans/disconnected-rebuild/environments/dc-lab/dns-records.md
• ADRs 0005 (rebuild network/ingress/PKI), 0026 (IPv6 baseline)
• PowerDNS Authoritative docs: doc.powerdns.com/authoritative
• PowerDNS Recursor docs: doc.powerdns.com/recursor