The vault was 155 pages / 31,210 lines, most of it measurement ledgers for payloads that no longer run. A throughput figure for a runtime nobody operates is not a record: AI moves faster than the vault, so the number is wrong before anyone reads it, and a wrong number in a runbook gets believed. Per the new purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed. Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest): - the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM, the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two "Running Qwen3.6 …" pages; - the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag) and the multi-device/x4-pair/hybrid/offload experiment pages; - every vLLM page — no vLLM endpoint exists on this fleet; - `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff, self-hosted app census, the TTS survey) that were obsolete by construction; - `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the page that owns the thing it governs, never to a diary. Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras tables keep the window, payload, runtime, reason and rollback pointer; the rate columns are blanked. The live shapes stay verbatim — those numbers are what the deployer and `models.yml` consume.
279 lines
15 KiB
Markdown
279 lines
15 KiB
Markdown
---
|
|
type: reference
|
|
description: "The homelab gives every service a name under `franksdank.cloud`."
|
|
tags:
|
|
- infrastructure
|
|
- dns
|
|
timestamp: 2026-09-02
|
|
---
|
|
|
|
# LAN DNS
|
|
|
|
The homelab gives every service a name under `franksdank.cloud`. Users type a name; they never type a port number. This page covers how names reach users: the Pi-hole server, the G36 router hand-off, and the split between LAN and WAN resolution.
|
|
|
|
See [[Caddy]] for the reverse proxy that serves the names, and [[Network Architecture]] for the full topology.
|
|
|
|
## Components
|
|
|
|
| Component | Where | Role |
|
|
|-----------|-------|------|
|
|
| Pi-hole | Dankclaw quadlet (`dankclaw/pihole/`) | Answers the flat service names for LAN clients |
|
|
| SURFboard G36 | 192.168.0.1 | Hands dnsmasq to clients as the DHCP DNS server |
|
|
| Porkbun | Registrar for `franksdank.cloud` | Answers the **public** names for the WAN; internal-only names have no public record and resolve only on the LAN |
|
|
| Caddy | Dankclaw (192.168.0.143:443) | Serves every name over HTTPS |
|
|
|
|
## How a name reaches a client
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant C as LAN client (laptop, phone)
|
|
participant G as G36 router (192.168.0.1)
|
|
participant D as Pi-hole (192.168.0.143:53)
|
|
participant P as Porkbun (WAN)
|
|
participant X as Caddy (192.168.0.143:443)
|
|
|
|
C->>G: DHCP request
|
|
G-->>C: lease + DNS = 192.168.0.143
|
|
C->>D: dig romm.franksdank.cloud
|
|
D-->>C: 192.168.0.143
|
|
Note over D: addn-hosts answer<br/>(dankclaw/pihole/addn-hosts)
|
|
C->>X: https://romm.franksdank.cloud
|
|
X-->>C: RomM (192.168.0.235:8085 upstream)
|
|
C->>P: dig franksdank.cloud (unknown name)
|
|
P-->>C: 71.227.227.172
|
|
```
|
|
|
|
Pi-hole answers only the names in `addn-hosts`. Every other query goes to the upstream resolvers `1.1.1.1` and `1.0.0.1`. The zone apex and unknown subdomains fall through to Porkbun. This avoids a wildcard override.
|
|
|
|
## Why split DNS
|
|
|
|
The G36 has no NAT loopback (hairpin). A LAN client that resolves a name to the WAN IP and then connects to it would time out. Split DNS removes the hairpin: LAN clients get the LAN IP, WAN clients get the public IP. Both use the same name and the same Let's Encrypt certificate. See [[Caddy]] for the TLS side.
|
|
|
|
## Pi-hole quadlet
|
|
|
|
Pi-hole FTL is a dnsmasq fork. It serves the same `addn-hosts` file the retired dnsmasq quadlet used, mounted into `/etc/pihole/hosts/` read-only. It retired the dnsmasq quadlet on 2026-08-24.
|
|
|
|
- **Location:** `dankclaw/pihole/` — `quadlet/pihole.container`, `env.example`, `addn-hosts`
|
|
- **Ports:** 53/udp + 53/tcp (DNS) and 8083/tcp (web UI) on 192.168.0.143
|
|
- **Host prerequisite 1:** `sysctl net.ipv4.ip_unprivileged_port_start=53` (persisted in `/etc/sysctl.d/99-dnsmasq.conf`)
|
|
- **Host prerequisite 2:** `DNSStubListener=no` in `/etc/systemd/resolved.conf` (the resolved stub would otherwise hold port 53 inside the container netns)
|
|
- **Env file:** `/home/dank/Containers/pihole/env` (never committed) — `ServerIP`, `TZ`, `DNS1` + `DNS2` (pinned upstreams `1.1.1.1` + `1.0.0.1`), `WEBPASSWORD` (the shared family password)
|
|
- **Web UI:** `pihole.franksdank.cloud` (Caddy vhost → 8083), family login
|
|
- **Deploy + restart after a change:** `./scripts/deploy-pihole-quadlet.sh dankclaw`
|
|
|
|
Adding a new service name is a two-or-three step decision — decide the exposure
|
|
first, because it decides whether the name gets a public record at all:
|
|
|
|
1. `dankclaw/pihole/addn-hosts` — `192.168.0.143 <name>.franksdank.cloud` (always).
|
|
2. **Public** name only: Porkbun A record `<name> 71.227.227.172`, so WAN clients
|
|
resolve it too. An **internal-only** name deliberately gets no public record:
|
|
the query falls through to the upstream resolvers, which answer NXDOMAIN for
|
|
`franksdank.cloud` subdomains, so off-LAN clients cannot reach the vhost at all.
|
|
3. Add the vhost to [[Caddy]] and run `./scripts/deploy.sh dankclaw`. See
|
|
[[Pi-hole]] for the full configuration.
|
|
|
|
Internal-only names as of 2026-09-02: `prometheus.`, `loki.`, `grafana.`
|
|
(observability) and `seerr.` (the `*arr` request front-end). Their public A
|
|
records were deleted from the Porkbun zone that day; each is still served by
|
|
Caddy on the same name, so LAN clients see no change and WAN clients get
|
|
NXDOMAIN. DNS-01 issuance is unaffected — Caddy's challenge is a TXT record in
|
|
the same zone and needs no A record at all, which is why an internal-only vhost
|
|
still gets a real Let's Encrypt certificate.
|
|
|
|
## G36 router DNS hand-off
|
|
|
|
The router must tell DHCP clients to use Pi-hole. Until that step is done, new clients get the router's own DNS relay and the flat names do not resolve for them. Existing clients that already use Pi-hole are unaffected.
|
|
|
|
**Manual step (web GUI at `https://192.168.0.1`):**
|
|
|
|
1. Open **Gateway → Connection → Local IP Network → IPv4**.
|
|
2. Uncheck **DNS Proxy** (disable the router DNS relay).
|
|
3. Set **DNS** to **Statically configured**.
|
|
4. Set **Primary DNS** to `192.168.0.143`.
|
|
5. Leave the secondary field empty or set `0.0.0.0`.
|
|
6. Apply, then re-lease DHCP on each client (`sudo dhclient -r && sudo dhclient`).
|
|
|
|
The router admin interface has no programmatic API in this firmware. The web GUI is the only control path. Keep the credentials out of this repo; the repo tracks no router secrets.
|
|
|
|
Verify after the change:
|
|
|
|
```
|
|
# on a LAN client
|
|
resolv.conf # nameserver 192.168.0.143
|
|
dig romm.franksdank.cloud +short
|
|
# expected: 192.168.0.143
|
|
```
|
|
|
|
## Dankclaw host DNS
|
|
|
|
The Dankclaw host itself resolves the flat names through Pi-hole:
|
|
|
|
- `/etc/systemd/resolved.conf`: `DNS=192.168.0.143 192.168.0.1` (Pi-hole first, router as fallback)
|
|
|
|
Without this, the host would resolve the names through the router to the WAN IP and time out (no hairpin).
|
|
|
|
## Dankware host DNS
|
|
|
|
`dankware` is a LAN client, not a Pi-hole server. Its flat names resolve through Pi-hole (`192.168.0.143`), but its resolver setup differs from Dankclaw's because it carries two `+DefaultRoute` links.
|
|
|
|
- **Two default links:** `enp47s0` (ethernet, metric 100) and `wlan0` (wifi, metric 600), each getting the same DHCP DNS list `[192.168.0.143, 1.1.1.1, …]`.
|
|
- **The flap:** `systemd-resolved` picks a different "current" resolver per link — `enp47s0` settled on the Comcast v6 upstream (→ WAN IP, times out on the LAN), `wlan0` on Pi-hole (→ LAN IP). Flat-name resolution flapped between the two on every query.
|
|
- **The fix:** pin `192.168.0.143` as the static IPv4 resolver on both NetworkManager connections. `resolved.conf`'s own `DNS=` is a per-link fallback, so the link itself must be pinned:
|
|
|
|
```bash
|
|
for c in "Wired connection 1" FRANKFI-5G; do
|
|
nmcli connection modify "$c" ipv4.method auto \
|
|
ipv4.dns 192.168.0.143 \
|
|
ipv4.ignore-auto-dns yes \
|
|
ipv4.ignore-auto-routes no
|
|
done
|
|
nmcli connection up "Wired connection 1"
|
|
nmcli connection up FRANKFI-5G
|
|
systemctl restart systemd-resolved
|
|
```
|
|
|
|
`method:auto` keeps DHCP for the address/gateway/route (dual-default traffic path preserved); `ignore-auto-dns=yes` uses only the static resolver; `ignore-auto-routes=no` keeps the DHCP routes. Verify with a cache-clear loop — `romm.franksdank.cloud` should return `192.168.0.143` on every run:
|
|
|
|
```bash
|
|
for i in $(seq 1 10); do resolvectl flush-caches; dig romm.franksdank.cloud +short; done
|
|
```
|
|
|
|
The apex and unknown subdomains still fall through to Porkbun (→ `71.227.227.172`), so WAN clients and a bare `dig franksdank.cloud` keep working.
|
|
|
|
## WAN records (Porkbun)
|
|
|
|
The Porkbun zone `franksdank.cloud` carries an A record for every vhost plus the apex, all pointing at `71.227.227.172`. TTL is 300. The zone has no wildcard record: unknown subdomains return NXDOMAIN. Caddy's DNS-01 challenges add and remove `_acme-challenge.*` TXT records through the same API during issuance. See [[Caddy]] for the TLS flow.
|
|
|
|
`franksdank.casa` is not in this Porkbun account. The `plex.franksdank.casa` vhost stays plain HTTP for that reason. The canonical Plex name is `plex.franksdank.cloud` (HTTPS).
|
|
|
|
## Failure modes
|
|
|
|
| Symptom | First check |
|
|
|---------|-------------|
|
|
| Name does not resolve on a new client | `resolv.conf` shows 192.168.0.1, not .143 — the G36 hand-off step is missing |
|
|
| Name resolves to 71.227.x and times out on the LAN | Client used the router DNS — same root cause |
|
|
| Flat name flaps between 192.168.0.143 and 71.227.x on one client | Two `+DefaultRoute` links; resolved picks a different "current" resolver per link. Pin 192.168.0.143 as static DNS on both NM connections — see [[#Dankware host DNS]] |
|
|
| `dig @192.168.0.143` fails entirely | Pi-hole quadlet down: `systemctl --user status pihole` |
|
|
| Single-label name (for example `dankclaw`) answers `127.0.1.1` | The rootless container `/etc/hosts` carries the host entry (`127.0.1.1 <host>`). FTL serves it. Use the FQDN, or an `/etc/hosts` entry on the client |
|
|
| HTTPS works from the host but not the WAN | No port forward on the G36 for that service (manual router step) |
|
|
| Cert renewal fails in Caddy logs | Porkbun API credentials in the Dankclaw `.env`; test with the Porkbun API `/ping` endpoint |
|
|
|
|
## Per-client fix: resolver drop-in + browser DoH bypass
|
|
|
|
Symptom: a LAN machine gets `connection timeout` (curl rc=28) on
|
|
`https://*.franksdank.cloud` while the server is healthy. Root cause: the
|
|
public DNS hands out the WAN IP `71.227.227.172`, which has **no 443
|
|
port-forward** (filtered). Any resolver that is NOT Pi-hole produces the WAN
|
|
answer → SYN to a filtered port → timeout. There are TWO resolver layers to
|
|
check, in order:
|
|
|
|
**Layer 1 — systemd-resolved (fixes curl/shell).** A route-only drop-in
|
|
sends `*.franksdank.cloud` ONLY to Pi-hole and never to the default-route
|
|
servers:
|
|
|
|
```bash
|
|
dig +short www.franksdank.cloud @192.168.0.143 # expect 192.168.0.143
|
|
dig +short www.franksdank.cloud @1.1.1.1 # expect 71.227.227.172 (the trap)
|
|
getent hosts www.franksdank.cloud # what the OS actually resolves
|
|
curl -sk --resolve www.franksdank.cloud:443:192.168.0.143 \
|
|
https://www.franksdank.cloud/ -o /dev/null -w '%{http_code}\n' # 200 = server fine
|
|
|
|
sudo mkdir -p /etc/systemd/resolved.conf.d
|
|
sudo tee /etc/systemd/resolved.conf.d/franksdank-lan.conf >/dev/null <<'EOF'
|
|
[Resolve]
|
|
DNS=192.168.0.143
|
|
Domains=~franksdank.cloud ~franksdank.casa
|
|
EOF
|
|
sudo systemctl restart systemd-resolved && resolvectl flush-caches
|
|
```
|
|
|
|
The `~` (route-only) prefix is load-bearing: it routes only those domains to
|
|
Pi-hole. Verify: `getent hosts www.franksdank.cloud` → `192.168.0.143`
|
|
`resolvectl status` shows `DNS Domain: ~franksdank.cloud ~franksdank.casa`.
|
|
|
|
**Layer 2 — browser DoH (the bypass: "curl works, browser times out").** If
|
|
Layer 1 is verified but the BROWSER still times out, the browser is doing
|
|
its own DNS over HTTPS and never asks systemd-resolved — it queries the TRR
|
|
provider (Cloudflare et al.) directly → WAN answer → timeout. Prove it:
|
|
`curl -sk -m 6 https://71.227.227.172/` → rc=28 (the WAN answer is a dead
|
|
end). Diagnosis details:
|
|
|
|
- Firefox profile may be at the XDG path `~/.config/mozilla/firefox/`, NOT
|
|
`~/.mozilla/firefox/` — find it via the running process if the default
|
|
path is empty.
|
|
- `grep network.trr. <profile>/prefs.js` — **no entries = shipped default,
|
|
and since Firefox 113 the shipped default is DoH ON.**
|
|
|
|
Fix (surgical, keeps DoH for everything else): Firefox → Settings → Privacy
|
|
& Security → **DNS over HTTPS** → "enabled for these domains" → **Add…** →
|
|
`franksdank.cloud` (and `franksdank.casa`). The exception list = domains
|
|
that do NOT use DoH; they fall back to the system resolver, which has the
|
|
route-only drop-in. Or set DoH to **Off** entirely (coarser). Chrome:
|
|
Settings → Privacy and security → Security → "Use secure DNS" → off, or
|
|
"Use the same settings as your system". Do NOT hand-edit `prefs.js` while
|
|
the browser is running — it holds prefs in memory and rewrites the file on
|
|
exit, clobbering the edit.
|
|
|
|
This is a per-client fix: every LAN client that must reach the homelab by
|
|
name needs Pi-hole as its DNS (or the drop-in) AND a browser that does not
|
|
bypass it with DoH. The server side needs no change. Verified on blankpad
|
|
2026-09-21: the drop-in was present since 09-12, curl returned 200, and
|
|
Firefox 155 with default (ON) DoH was the remaining hole.
|
|
|
|
## Porkbun zone ops (franksdank.cloud)
|
|
|
|
Operate the public DNS zone via the Porkbun API v3, run **on Dankclaw** so
|
|
the API key/secret sourced from `/home/dank/serverAppConfigs/.env`
|
|
(`PORKBUN_API_KEY`, `PORKBUN_API_SECRET_KEY`) never leaves the host. Pipe
|
|
scripts to the target with `ssh -o BatchMode=yes dankclaw 'bash -s' <
|
|
/tmp/script.sh`.
|
|
|
|
Endpoints (all POST, JSON body):
|
|
|
|
- List: `https://api.porkbun.com/api/json/v3/dns/retrieve/franksdank.cloud`
|
|
— auth via headers `X-API-Key` + `X-Secret-API-Key` (Bearer is wrong
|
|
here). Response shape is **top-level `.records[]`**, NOT
|
|
`.content.records[]` as some docs claim — parse with
|
|
`jq -r '.records[] | [.id,.name,.type,.content,(.ttl|tostring)] | @tsv'`.
|
|
- Delete: `/dns/delete/{domain}/{id}` (body carries apikey+secretapikey).
|
|
- Edit: `/dns/edit/{domain}/{id}` — pass the full record (`id`, `name`
|
|
`type`, `content`, `ttl`, `prio`).
|
|
|
|
Quirks that bite (verified live 2026-08-31):
|
|
|
|
1. **`name` is a RELATIVE label on edit.** Passing `ai.franksdank.cloud`
|
|
yields the record `ai.franksdank.cloud.franksdank.cloud`. Pass the bare
|
|
label (`ai`) and the API appends the zone itself. Verify after every
|
|
rename with a retrieve before declaring done.
|
|
2. **Rate limit answers as an HTML 404-style page, not JSON.** A batch of
|
|
rapid mutations makes some calls return `<!DOCTYPE html>...` where jq
|
|
then errors (`Cannot iterate over null` / `Invalid numeric literal`). Do
|
|
not assume the call succeeded — sleep ~8 s and retry the failed ids;
|
|
verify by re-listing the zone at the end.
|
|
3. Deletes/renames are per-record-id: always retrieve first, map name→id,
|
|
then mutate by id.
|
|
|
|
Rename procedure (vhost rename = DNS + Caddy together):
|
|
|
|
1. `retrieve` → collect ids for the renamed names.
|
|
2. `edit` each id with the new **relative** label, preserving
|
|
type/content/ttl.
|
|
3. Re-retrieve and confirm `<new>.franksdank.cloud A <ip>`.
|
|
4. In-repo lockstep before deploy: rename the Caddyfile vhost block AND the
|
|
Pi-hole `dankclaw/pihole/addn-hosts` line in the same commit — a test
|
|
fails the suite if any Caddyfile vhost lacks an addn-hosts A record on
|
|
192.168.0.143.
|
|
5. Deploy via `scripts/deploy.sh dankclaw` (plans pihole+caddy from changed
|
|
paths). Capture the suite's true exit with
|
|
`out=$(./tests/run.sh 2>&1); rc=$?` — piping to `tail` masks the exit
|
|
code behind `tail`'s own status and has shipped untested commits here.
|
|
6. Verify live: `dig +short <name> @192.168.0.143` then
|
|
`curl -o /dev/null -w %{http_code} https://<new-name>` (expect 200/302;
|
|
first hit triggers dns-01 issuance, which works without inbound port 80).
|
|
|
|
## See Also
|
|
|
|
- [[Pi-hole]] — the LAN DNS server (quadlet, records, web UI)
|
|
- [[Caddy]] — TLS, vhost list, DNS-01
|
|
- [[Network Architecture]] — topology, access policy
|
|
- [[Dankclaw Stack]] — the host that carries Pi-hole and Caddy |