Files
Frank 327e78b950 docs(vault): destroy the dead weight — 53 pages and the change diary
The vault was 155 pages / 31,210 lines, most of it measurement ledgers for
payloads that no longer run. A throughput figure for a runtime nobody operates is
not a record: AI moves faster than the vault, so the number is wrong before
anyone reads it, and a wrong number in a runbook gets believed. Per the new
purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed.

Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest):
- the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM,
  the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch
  campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two
  "Running Qwen3.6 …" pages;
- the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag)
  and the multi-device/x4-pair/hybrid/offload experiment pages;
- every vLLM page — no vLLM endpoint exists on this fleet;
- `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced
  owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff,
  self-hosted app census, the TTS survey) that were obsolete by construction;
- `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the
  page that owns the thing it governs, never to a diary.

Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras
tables keep the window, payload, runtime, reason and rollback pointer; the rate
columns are blanked. The live shapes stay verbatim — those numbers are what the
deployer and `models.yml` consume.
2026-10-19 10:56:37 -07:00

279 lines
15 KiB
Markdown

---
type: reference
description: "The homelab gives every service a name under `franksdank.cloud`."
tags:
- infrastructure
- dns
timestamp: 2026-09-02
---
# LAN DNS
The homelab gives every service a name under `franksdank.cloud`. Users type a name; they never type a port number. This page covers how names reach users: the Pi-hole server, the G36 router hand-off, and the split between LAN and WAN resolution.
See [[Caddy]] for the reverse proxy that serves the names, and [[Network Architecture]] for the full topology.
## Components
| Component | Where | Role |
|-----------|-------|------|
| Pi-hole | Dankclaw quadlet (`dankclaw/pihole/`) | Answers the flat service names for LAN clients |
| SURFboard G36 | 192.168.0.1 | Hands dnsmasq to clients as the DHCP DNS server |
| Porkbun | Registrar for `franksdank.cloud` | Answers the **public** names for the WAN; internal-only names have no public record and resolve only on the LAN |
| Caddy | Dankclaw (192.168.0.143:443) | Serves every name over HTTPS |
## How a name reaches a client
```mermaid
sequenceDiagram
participant C as LAN client (laptop, phone)
participant G as G36 router (192.168.0.1)
participant D as Pi-hole (192.168.0.143:53)
participant P as Porkbun (WAN)
participant X as Caddy (192.168.0.143:443)
C->>G: DHCP request
G-->>C: lease + DNS = 192.168.0.143
C->>D: dig romm.franksdank.cloud
D-->>C: 192.168.0.143
Note over D: addn-hosts answer<br/>(dankclaw/pihole/addn-hosts)
C->>X: https://romm.franksdank.cloud
X-->>C: RomM (192.168.0.235:8085 upstream)
C->>P: dig franksdank.cloud (unknown name)
P-->>C: 71.227.227.172
```
Pi-hole answers only the names in `addn-hosts`. Every other query goes to the upstream resolvers `1.1.1.1` and `1.0.0.1`. The zone apex and unknown subdomains fall through to Porkbun. This avoids a wildcard override.
## Why split DNS
The G36 has no NAT loopback (hairpin). A LAN client that resolves a name to the WAN IP and then connects to it would time out. Split DNS removes the hairpin: LAN clients get the LAN IP, WAN clients get the public IP. Both use the same name and the same Let's Encrypt certificate. See [[Caddy]] for the TLS side.
## Pi-hole quadlet
Pi-hole FTL is a dnsmasq fork. It serves the same `addn-hosts` file the retired dnsmasq quadlet used, mounted into `/etc/pihole/hosts/` read-only. It retired the dnsmasq quadlet on 2026-08-24.
- **Location:** `dankclaw/pihole/` — `quadlet/pihole.container`, `env.example`, `addn-hosts`
- **Ports:** 53/udp + 53/tcp (DNS) and 8083/tcp (web UI) on 192.168.0.143
- **Host prerequisite 1:** `sysctl net.ipv4.ip_unprivileged_port_start=53` (persisted in `/etc/sysctl.d/99-dnsmasq.conf`)
- **Host prerequisite 2:** `DNSStubListener=no` in `/etc/systemd/resolved.conf` (the resolved stub would otherwise hold port 53 inside the container netns)
- **Env file:** `/home/dank/Containers/pihole/env` (never committed) — `ServerIP`, `TZ`, `DNS1` + `DNS2` (pinned upstreams `1.1.1.1` + `1.0.0.1`), `WEBPASSWORD` (the shared family password)
- **Web UI:** `pihole.franksdank.cloud` (Caddy vhost → 8083), family login
- **Deploy + restart after a change:** `./scripts/deploy-pihole-quadlet.sh dankclaw`
Adding a new service name is a two-or-three step decision — decide the exposure
first, because it decides whether the name gets a public record at all:
1. `dankclaw/pihole/addn-hosts` — `192.168.0.143 <name>.franksdank.cloud` (always).
2. **Public** name only: Porkbun A record `<name> 71.227.227.172`, so WAN clients
resolve it too. An **internal-only** name deliberately gets no public record:
the query falls through to the upstream resolvers, which answer NXDOMAIN for
`franksdank.cloud` subdomains, so off-LAN clients cannot reach the vhost at all.
3. Add the vhost to [[Caddy]] and run `./scripts/deploy.sh dankclaw`. See
[[Pi-hole]] for the full configuration.
Internal-only names as of 2026-09-02: `prometheus.`, `loki.`, `grafana.`
(observability) and `seerr.` (the `*arr` request front-end). Their public A
records were deleted from the Porkbun zone that day; each is still served by
Caddy on the same name, so LAN clients see no change and WAN clients get
NXDOMAIN. DNS-01 issuance is unaffected — Caddy's challenge is a TXT record in
the same zone and needs no A record at all, which is why an internal-only vhost
still gets a real Let's Encrypt certificate.
## G36 router DNS hand-off
The router must tell DHCP clients to use Pi-hole. Until that step is done, new clients get the router's own DNS relay and the flat names do not resolve for them. Existing clients that already use Pi-hole are unaffected.
**Manual step (web GUI at `https://192.168.0.1`):**
1. Open **Gateway → Connection → Local IP Network → IPv4**.
2. Uncheck **DNS Proxy** (disable the router DNS relay).
3. Set **DNS** to **Statically configured**.
4. Set **Primary DNS** to `192.168.0.143`.
5. Leave the secondary field empty or set `0.0.0.0`.
6. Apply, then re-lease DHCP on each client (`sudo dhclient -r && sudo dhclient`).
The router admin interface has no programmatic API in this firmware. The web GUI is the only control path. Keep the credentials out of this repo; the repo tracks no router secrets.
Verify after the change:
```
# on a LAN client
resolv.conf # nameserver 192.168.0.143
dig romm.franksdank.cloud +short
# expected: 192.168.0.143
```
## Dankclaw host DNS
The Dankclaw host itself resolves the flat names through Pi-hole:
- `/etc/systemd/resolved.conf`: `DNS=192.168.0.143 192.168.0.1` (Pi-hole first, router as fallback)
Without this, the host would resolve the names through the router to the WAN IP and time out (no hairpin).
## Dankware host DNS
`dankware` is a LAN client, not a Pi-hole server. Its flat names resolve through Pi-hole (`192.168.0.143`), but its resolver setup differs from Dankclaw's because it carries two `+DefaultRoute` links.
- **Two default links:** `enp47s0` (ethernet, metric 100) and `wlan0` (wifi, metric 600), each getting the same DHCP DNS list `[192.168.0.143, 1.1.1.1, …]`.
- **The flap:** `systemd-resolved` picks a different "current" resolver per link — `enp47s0` settled on the Comcast v6 upstream (→ WAN IP, times out on the LAN), `wlan0` on Pi-hole (→ LAN IP). Flat-name resolution flapped between the two on every query.
- **The fix:** pin `192.168.0.143` as the static IPv4 resolver on both NetworkManager connections. `resolved.conf`'s own `DNS=` is a per-link fallback, so the link itself must be pinned:
```bash
for c in "Wired connection 1" FRANKFI-5G; do
nmcli connection modify "$c" ipv4.method auto \
ipv4.dns 192.168.0.143 \
ipv4.ignore-auto-dns yes \
ipv4.ignore-auto-routes no
done
nmcli connection up "Wired connection 1"
nmcli connection up FRANKFI-5G
systemctl restart systemd-resolved
```
`method:auto` keeps DHCP for the address/gateway/route (dual-default traffic path preserved); `ignore-auto-dns=yes` uses only the static resolver; `ignore-auto-routes=no` keeps the DHCP routes. Verify with a cache-clear loop — `romm.franksdank.cloud` should return `192.168.0.143` on every run:
```bash
for i in $(seq 1 10); do resolvectl flush-caches; dig romm.franksdank.cloud +short; done
```
The apex and unknown subdomains still fall through to Porkbun (→ `71.227.227.172`), so WAN clients and a bare `dig franksdank.cloud` keep working.
## WAN records (Porkbun)
The Porkbun zone `franksdank.cloud` carries an A record for every vhost plus the apex, all pointing at `71.227.227.172`. TTL is 300. The zone has no wildcard record: unknown subdomains return NXDOMAIN. Caddy's DNS-01 challenges add and remove `_acme-challenge.*` TXT records through the same API during issuance. See [[Caddy]] for the TLS flow.
`franksdank.casa` is not in this Porkbun account. The `plex.franksdank.casa` vhost stays plain HTTP for that reason. The canonical Plex name is `plex.franksdank.cloud` (HTTPS).
## Failure modes
| Symptom | First check |
|---------|-------------|
| Name does not resolve on a new client | `resolv.conf` shows 192.168.0.1, not .143 — the G36 hand-off step is missing |
| Name resolves to 71.227.x and times out on the LAN | Client used the router DNS — same root cause |
| Flat name flaps between 192.168.0.143 and 71.227.x on one client | Two `+DefaultRoute` links; resolved picks a different "current" resolver per link. Pin 192.168.0.143 as static DNS on both NM connections — see [[#Dankware host DNS]] |
| `dig @192.168.0.143` fails entirely | Pi-hole quadlet down: `systemctl --user status pihole` |
| Single-label name (for example `dankclaw`) answers `127.0.1.1` | The rootless container `/etc/hosts` carries the host entry (`127.0.1.1 <host>`). FTL serves it. Use the FQDN, or an `/etc/hosts` entry on the client |
| HTTPS works from the host but not the WAN | No port forward on the G36 for that service (manual router step) |
| Cert renewal fails in Caddy logs | Porkbun API credentials in the Dankclaw `.env`; test with the Porkbun API `/ping` endpoint |
## Per-client fix: resolver drop-in + browser DoH bypass
Symptom: a LAN machine gets `connection timeout` (curl rc=28) on
`https://*.franksdank.cloud` while the server is healthy. Root cause: the
public DNS hands out the WAN IP `71.227.227.172`, which has **no 443
port-forward** (filtered). Any resolver that is NOT Pi-hole produces the WAN
answer → SYN to a filtered port → timeout. There are TWO resolver layers to
check, in order:
**Layer 1 — systemd-resolved (fixes curl/shell).** A route-only drop-in
sends `*.franksdank.cloud` ONLY to Pi-hole and never to the default-route
servers:
```bash
dig +short www.franksdank.cloud @192.168.0.143 # expect 192.168.0.143
dig +short www.franksdank.cloud @1.1.1.1 # expect 71.227.227.172 (the trap)
getent hosts www.franksdank.cloud # what the OS actually resolves
curl -sk --resolve www.franksdank.cloud:443:192.168.0.143 \
https://www.franksdank.cloud/ -o /dev/null -w '%{http_code}\n' # 200 = server fine
sudo mkdir -p /etc/systemd/resolved.conf.d
sudo tee /etc/systemd/resolved.conf.d/franksdank-lan.conf >/dev/null <<'EOF'
[Resolve]
DNS=192.168.0.143
Domains=~franksdank.cloud ~franksdank.casa
EOF
sudo systemctl restart systemd-resolved && resolvectl flush-caches
```
The `~` (route-only) prefix is load-bearing: it routes only those domains to
Pi-hole. Verify: `getent hosts www.franksdank.cloud` → `192.168.0.143`
`resolvectl status` shows `DNS Domain: ~franksdank.cloud ~franksdank.casa`.
**Layer 2 — browser DoH (the bypass: "curl works, browser times out").** If
Layer 1 is verified but the BROWSER still times out, the browser is doing
its own DNS over HTTPS and never asks systemd-resolved — it queries the TRR
provider (Cloudflare et al.) directly → WAN answer → timeout. Prove it:
`curl -sk -m 6 https://71.227.227.172/` → rc=28 (the WAN answer is a dead
end). Diagnosis details:
- Firefox profile may be at the XDG path `~/.config/mozilla/firefox/`, NOT
`~/.mozilla/firefox/` — find it via the running process if the default
path is empty.
- `grep network.trr. <profile>/prefs.js` — **no entries = shipped default,
and since Firefox 113 the shipped default is DoH ON.**
Fix (surgical, keeps DoH for everything else): Firefox → Settings → Privacy
& Security → **DNS over HTTPS** → "enabled for these domains" → **Add…** →
`franksdank.cloud` (and `franksdank.casa`). The exception list = domains
that do NOT use DoH; they fall back to the system resolver, which has the
route-only drop-in. Or set DoH to **Off** entirely (coarser). Chrome:
Settings → Privacy and security → Security → "Use secure DNS" → off, or
"Use the same settings as your system". Do NOT hand-edit `prefs.js` while
the browser is running — it holds prefs in memory and rewrites the file on
exit, clobbering the edit.
This is a per-client fix: every LAN client that must reach the homelab by
name needs Pi-hole as its DNS (or the drop-in) AND a browser that does not
bypass it with DoH. The server side needs no change. Verified on blankpad
2026-09-21: the drop-in was present since 09-12, curl returned 200, and
Firefox 155 with default (ON) DoH was the remaining hole.
## Porkbun zone ops (franksdank.cloud)
Operate the public DNS zone via the Porkbun API v3, run **on Dankclaw** so
the API key/secret sourced from `/home/dank/serverAppConfigs/.env`
(`PORKBUN_API_KEY`, `PORKBUN_API_SECRET_KEY`) never leaves the host. Pipe
scripts to the target with `ssh -o BatchMode=yes dankclaw 'bash -s' <
/tmp/script.sh`.
Endpoints (all POST, JSON body):
- List: `https://api.porkbun.com/api/json/v3/dns/retrieve/franksdank.cloud`
— auth via headers `X-API-Key` + `X-Secret-API-Key` (Bearer is wrong
here). Response shape is **top-level `.records[]`**, NOT
`.content.records[]` as some docs claim — parse with
`jq -r '.records[] | [.id,.name,.type,.content,(.ttl|tostring)] | @tsv'`.
- Delete: `/dns/delete/{domain}/{id}` (body carries apikey+secretapikey).
- Edit: `/dns/edit/{domain}/{id}` — pass the full record (`id`, `name`
`type`, `content`, `ttl`, `prio`).
Quirks that bite (verified live 2026-08-31):
1. **`name` is a RELATIVE label on edit.** Passing `ai.franksdank.cloud`
yields the record `ai.franksdank.cloud.franksdank.cloud`. Pass the bare
label (`ai`) and the API appends the zone itself. Verify after every
rename with a retrieve before declaring done.
2. **Rate limit answers as an HTML 404-style page, not JSON.** A batch of
rapid mutations makes some calls return `<!DOCTYPE html>...` where jq
then errors (`Cannot iterate over null` / `Invalid numeric literal`). Do
not assume the call succeeded — sleep ~8 s and retry the failed ids;
verify by re-listing the zone at the end.
3. Deletes/renames are per-record-id: always retrieve first, map name→id,
then mutate by id.
Rename procedure (vhost rename = DNS + Caddy together):
1. `retrieve` → collect ids for the renamed names.
2. `edit` each id with the new **relative** label, preserving
type/content/ttl.
3. Re-retrieve and confirm `<new>.franksdank.cloud A <ip>`.
4. In-repo lockstep before deploy: rename the Caddyfile vhost block AND the
Pi-hole `dankclaw/pihole/addn-hosts` line in the same commit — a test
fails the suite if any Caddyfile vhost lacks an addn-hosts A record on
192.168.0.143.
5. Deploy via `scripts/deploy.sh dankclaw` (plans pihole+caddy from changed
paths). Capture the suite's true exit with
`out=$(./tests/run.sh 2>&1); rc=$?` — piping to `tail` masks the exit
code behind `tail`'s own status and has shipped untested commits here.
6. Verify live: `dig +short <name> @192.168.0.143` then
`curl -o /dev/null -w %{http_code} https://<new-name>` (expect 200/302;
first hit triggers dns-01 issuance, which works without inbound port 80).
## See Also
- [[Pi-hole]] — the LAN DNS server (quadlet, records, web UI)
- [[Caddy]] — TLS, vhost list, DNS-01
- [[Network Architecture]] — topology, access policy
- [[Dankclaw Stack]] — the host that carries Pi-hole and Caddy