Compare commits

...
Author SHA1 Message Date
Frank d4ce76b68b refact(comments): state the mechanism, not the incident
Every comment that invoked a 'law' (50 word-boundary sites across 28
tracked files) now states what is going on and why, in terms that
transfer to any project: gate twins drift until one side greenlights
what the other refuses; a tag re-points while a digest pins; a
restrictive engine behavior belongs to the runtime that imposed it and
must be re-tested, not inherited; a speculative head is tensors in the
file, not a flag; snapshot links dangle where the container resolves
them; exit 0 is a clean stop to systemd, so a self-exiting daemon on
on-failure goes dark reporting success. Event narratives and dates are
gone from these comments; the mechanics carry the argument.

Behavior-identical: bash -n, py_compile, yaml parse, full bats suite
408/408 green. One symbol rename rides along (stack script local
scale_law -> scale_contract, with its log text).
2026-10-14 13:36:40 -07:00
Frank 65bd5ccea7 feat(rank6): node-exporter renders from a template; guard chain is one snippet
- service-map render node-exporter-unit --host H: the MIRROR pair comment
  ('keep this file in step') becomes a template. Per-host title, LAN
  address from the hosts map, port from the node-<host> scrape job. Both
  tracked quadlets are now rendered artifacts; check_drift gates them.
- dankclaw/Caddyfile: the nine verbatim Anubis pre-auth chains
  (reverse_proxy anubis:8923 + X-Real-Ip) collapse into the (guarded)
  named snippet; guarded vhosts are 'import guarded'. Validated with real
  caddy 2.11.7 (import resolution clean; the stock image stops at the
  porkbun module and the git basicauth placeholder, as designed).
- vhost-scoped guard greps (authentik, homepage_caddy, marreta_quadlet)
  now pin 'import guarded' in the vhost block.
- Service Inventory + Authentik docs carry the new laws.
2026-10-13 22:40:50 -07:00
Frank 5976ad0550 refact(docs): compose-era sweeps retire with the last compose file
rank 7 tail. dankserver/compose.yaml (a parallel inventory whose mounts
had diverged from the quadlets) and its .env.example live in
dankserver/.AGENTTRASH/20261013-compose-retire/; the retired ik chat
template joined scripts/.AGENTTRASH/. No target has a compose file now:
- service-map drops the compose selection rule, compose file claim, and
  check_compose (the check list is schema/units/ownership/config/drift);
- deploy-remote's compose fallback is a loud failure, not a WARN-and-
  nothing for dankserver;
- compose_validity.bats goes to tests/.AGENTTRASH (it was sweeping only
  the trash file); env_consistency sheds its two compose sweeps;
- docs_completeness's compose sweep skips .AGENTTRASH so retired
  services cannot resurrect, and a revived compose stays covered;
- Testing Policy drops the Compose validity row.
2026-10-13 22:35:28 -07:00
Frank 5ae6b7b928 refact(deploy-remote): shared root-unit start, config copy, retire helpers
rank 5. The three LLM handlers duplicated the same tail (relink,
daemon-reload, enable, restart, /health gate, MainPID port proof) ->
start_root_unit SERVICE PORT LIMIT; frank-archon's bespoke 900-iteration
loop was exactly wait_for_health at limit 900. copy_config DEST SRC...
replaces five hand-rolled mkdir+cp pairs (loki, alloy, grafana x2);
retire_unit_file PATH TRASH [UNIT] replaces the duplicated compose-era
unit retirements (halogen dragoon quadlet, comfyui.service). The health
grep seam now pins the per-endpoint call (start_root_unit frank-dragoon
7770).
2026-10-13 22:35:12 -07:00
Frank cc0bc06288 fix(pihole): addn-hosts renders no inline comments
dnsmasq strips ONLY whole-line comments in addn-hosts: a trailing
'# service' on a data line becomes a hostname token. The inventory's
Pi-hole renderer now emits pure IP-name lines (attribution lives in the
inventory) and the tracked file is re-rendered to match; pihole.bats
line-validity seams are green on the rendered artifact.
2026-10-13 22:35:12 -07:00
Frank bdead570e2 feat(inventory): one service inventory drives planner, allow-lists, and seams
rank 3. shared/services.yml is the single source of truth for every
deployable service (hosts, unit dirs, units, path claims, tests, scrape/
dns/probe facts, aliases, stacks); scripts/service-map.py is its only
reader (hosts/services/allows/plan/seams/owner/units/stack/get/render/
check). deploy-plan.py becomes a thin CLI wrapper; both --service
allow-lists (deploy.sh, deploy-remote.sh) and the changed-test mapper are
the inventory. Selection semantics: host scoping (a host's tree selects
only its own services), the quadlet netns/network coupling graph resolved
across directories (gluetun's riders live in their own dirs), stacks, and
canonical restart order.

Against the pinned old-planner fixture: 921 raw deltas reduced to 17, all
intentional (anubis now maps; cross-host paths select nothing; the retired
qwen-sharp template plans nothing and the live froggeric template plans
frank-archon; dankserver/romm/romm.container, missed by the old filename
rule, plans the romm trio). The fixture is regenerated as the new pin.

Integrity gates: schema, unit claims, quadlet-file ownership, path-glob
liveness, Pi-hole hosts / Prometheus jobs / DNS / homepage probes drift.
tests/service_inventory.bats pins all of it plus the plan fixture and the
deploy-plan CLI. Allow-list grep seams in hermes_webui/observability_quadlet
become behavioral allows calls.
2026-10-13 22:35:07 -07:00
Frank 156ea71879 fix(fleet): every LLM root unit restarts on a voluntary exit; parity seam gates it
frank-zealot shipped Restart=on-failure while its two siblings shipped
always. exit 0 is a CLEAN stop to systemd: on-failure leaves the endpoint
dark and the unit GREEN until the next boot — and frank-zealot is the
compaction hub for all three endpoints, so its quiet death degrades every
conversation in the fleet.

tests/llm_units.bats gains the fleet parity seam (Restart=always,
RestartSec 1..20, WantedBy=multi-user.target across scripts/frank-*.service);
the per-unit seams drop their weaker Restart greps so the policy lives in
one place. Runbook unit line updated.
2026-10-13 22:04:48 -07:00
Frank ab80c1366d refact(deployer): one MainPID gate for all three LLM endpoints
frank-dragoon's health gate never checked WHO answered :7770 — the
false-green the fleet learned about on 2026-09-26 (a stray probe holding
the port satisfies a bare curl while the unit is stopped). The other two
handlers carried that check as inlined copies; all three now call
assert_unit_owns_port, and dragoon gets the protection it was missing.

The seam follows: the endpoints' text-grep deployer tests are deleted,
because tests/llm_deploy_gates.bats EXECUTES the same handlers end to end
(stub systemctl/podman/sudo/ss/curl) and asserts refusals, restarts, unit
links, the pending marker, and the MainPID hardening. GATE_PREFIX is
exported at each handler for the same reason the serve scripts export it.
2026-10-13 22:02:48 -07:00
Frank c5a816e6a8 refact(deployer): the LLM handlers gate through the shared library
deploy_llm_dragoon/zealot/arkon now call require_image / require_bytes /
require_min_mib / require_file / require_dir_files, so the deployer and the
serve scripts refuse the same missing payload with the same words — the
2026-10-01 crash-loop law, enforced by construction instead of by duplicated
shell. require_exec joins the library for the native-runtime gate (a
compiled tree that lost its +x refuses like a missing one). relink_unit
takes SYSTEMD_SYSTEM_DIR so a handler can be driven end to end in a sandbox.

The library's own refusals now carry their status: a bash function that
ends inside an if returns the CONDITION's status, so a gate whose last
statement is its refusal printed the message and returned SUCCESS — the
new seam caught it, and the gate tests pin it.

New seam tests/llm_deploy_gates.bash + tests/stubs/: deploy-remote.sh's LLM
handlers EXECUTED against stub systemctl/podman/sudo/ss/curl, asserting
exit status, refusal text, no restart, and the pending state marker.
2026-10-13 21:58:13 -07:00
Frank cb25796a6b refact(frank-archon): serve gates run on the shared library; the serve seam is behavioural
The image/.rad/template gates are require_image/require_min_mib/require_file,
and the gates an operator can only reach on the host (GPU nodes, the ttm pages
cap, the run line) are now drivable: FRANK_ARCHON_DEV_KFD, FRANK_ARCHON_DEV_DRI,
FRANK_ARCHON_TTM_SYSFS and FRANK_ARCHON_LOG default to the production paths.
Seven new seam tests EXECUTE the serve script against fixtures (stub podman,
stub nodes, stub sysfs, temp payload) and assert exit status, the refusal
words and the engine argv. The old text-grep gate test is deleted: it pinned
the words of the source, not the behaviour.
2026-10-13 21:44:38 -07:00
Frank f31e7c5ed8 refact(frank-zealot): serve gates run on the shared gate library
The three -e gates (runtime, GGUF, mmproj) are require_file calls: the refusal
now names the artifact class, the exact path and the script that produces it.
The seam asserts that observable triple instead of the old sentence order.
2026-10-13 21:41:18 -07:00
Frank 028afdeb58 refact(frank-dragoon): serve gates run on the shared gate library
The hand-written image/ck/tokenizer gates are require_image, require_bytes and
require_dir_files from scripts/lib/llm-gates.sh. The serve seam is unchanged
and still green (its podman stub now models `podman image exists`, the
primitive the gate uses). Behaviour preserved: every refusal names the
artifact, the endpoint and the fetch script.
2026-10-13 21:40:02 -07:00
Frank acf2a5cee5 feat(scripts): one artifact-gate library for the LLM endpoints
The serve scripts and the deployer each hand-wrote the same missing/truncated
artifact gates; the 2026-10-01 crash-loop was those twins disagreeing
mid-cutover. scripts/lib/llm-gates.sh is the single place the refusal lives:
require_file / require_bytes / require_min_mib / require_dir_files /
require_image, each returning 1 and naming the endpoint, the artifact and the
remedy. New seam tests/llm_gates.bats pins the contract at the arguments-in /
status-and-stderr-out boundary.
2026-10-13 21:38:50 -07:00
65 changed files with 4866 additions and 1329 deletions
+28 -12
View File
@@ -35,11 +35,16 @@ the [runbook index](#runbook-index). The full swap history lives in
## Repository Layout
- `dankserver/` — compose.yaml + Caddyfile + .env.example for the media stack
- `dankclaw/` — compose.yaml + Caddyfile + .env.example for the AI stack
- `shared/` — homepage, Prometheus, Loki, Alloy, SearXNG configs
- `scripts/` — `deploy.sh` / `deploy-plan.py` / `deploy-remote.sh`, LLM serve
scripts, systemd units, env tunables, bench harnesses
- `dankserver/` — quadlet units + Caddyfile for the media stack (the
compose.yaml inventory retired 2026-10-13; its mounts had diverged from
the quadlets)
- `dankclaw/` — quadlet units + Caddyfile for the AI stack (compose retired 2026-09-16)
- `shared/` — the service inventory (`services.yml`), homepage, Prometheus,
Loki, Alloy, SearXNG configs
- `scripts/` — `deploy.sh` / `deploy-plan.py` / `deploy-remote.sh` and
`service-map.py` (the inventory's only reader), LLM serve scripts + the
shared artifact gates (`scripts/lib/llm-gates.sh`), systemd units, env
tunables, bench harnesses
- `old/` — retired LLM scripts (reference only)
- `serverAppConfigs-docs/` — knowledge base (Obsidian vault, OKF format).
The managed skill library is distilled there one page per skill-cluster:
@@ -70,8 +75,13 @@ All edits flow through git. Never edit files on a remote machine directly.
./scripts/deploy.sh dankclaw --dry-run
```
The planner maps changed paths to services; the remote runner restarts only
the selected service and records its revision outside the checkout.
The inventory (`shared/services.yml`, read only through
`scripts/service-map.py`) maps changed paths to services; the remote runner
restarts only the selected service and records its revision outside the
checkout. Both `--service` allow-lists, the planner, and the changed-test
seam mapper ARE the inventory — adding, moving, or retiring a service is a
data edit, not a script edit
(`serverAppConfigs-docs/Architecture/Service Inventory.md`).
Documentation-only changes are a no-op. LLM services are selected only by
their own files — every endpoint (`frank-dragoon`/`frank-zealot`/`frank-archon`)
by its unit/env/serve trio in `scripts/` — so
@@ -164,12 +174,17 @@ Tests are mandatory before commit (`bats` required on the dev machine):
./tests/run.sh
```
Seams: compose validity, env consistency (no leaked secrets), Caddyfile
Seams: env consistency (no leaked secrets), Caddyfile
validity, quadlet units, docs completeness **including vault link integrity**,
script syntax (shell + python), LLM unit integrity, LLM secret hygiene,
deployment flow, service-scoped planning, homepage config/pages/deploy/checker,
and `e2e_scaffold` — which fails if a Playwright spec file's number prefix is in
no project, because such a spec silently never runs.
LLM artifact gates (`llm_gates`/`llm_deploy_gates` — the shared
`scripts/lib/llm-gates.sh` every endpoint's serve and deploy gates run on),
deployment flow, service-scoped planning, the **service inventory**
(`service_inventory.bats`: inventory soundness, the pinned path->service
plan fixture, render drift for the Pi-hole hosts file), homepage
config/pages/deploy/checker, and `e2e_scaffold` — which fails if a
Playwright spec file's number prefix is in no project, because such a spec
silently never runs.
See `serverAppConfigs-docs/Architecture/Testing Policy.md`.
## Secret Rules
@@ -1041,7 +1056,7 @@ Active:
|---------|---------|
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
| frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
| frank-archon (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md` |
| frank-archon (radiance StillDeadcode, Swift1.5-Qwen3.8-Flash-Next GPTQ w4 + bf16-protected experts, int8 trunk, bf16 KV, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md` |
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
@@ -1050,6 +1065,7 @@ Retired (rollback targets and history):
| Runtime | Host | Window | Runbook |
|---------|------|--------|---------|
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
| radiance StillDeadcode Qwen3.8-Flash-Next native-FP8, 2×R9700 TP=2 (`frank-archon`) | Dankserver | 2026-10-08 → w4dn8 swap (rollback target — `.rad` payload on disk) | `frank-archon-radiance-qwen38-native-fp8.md` |
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
+24 -27
View File
@@ -56,6 +56,21 @@
}
}
# ═══════════════════════════════════════════════════════════════
# Named snippet: the pre-auth guard chain. Anubis (:8923) challenges,
# the embedded Authentik outpost authenticates, and authenticated
# traffic returns to the http://:8017 listener's post-auth handlers.
# Every guarded vhost is `import guarded` — the chain is EDITED HERE,
# never inlined per site (rank 6: it used to be nine verbatim copies).
# Exemptions live in their own vhosts (auth. is the IdP itself; git
# smart-HTTP and self-authenticating services document their reason).
# ═══════════════════════════════════════════════════════════════
(guarded) {
reverse_proxy anubis:8923 {
header_up X-Real-Ip {remote_host}
}
}
# --- Bare-IP access (health checks, convenience) ---
:80 {
redir https://www.franksdank.cloud/ 308
@@ -72,9 +87,7 @@ franksdank.cloud {
}
www.franksdank.cloud {
reverse_proxy anubis:8923 {
header_up X-Real-Ip {remote_host}
}
import guarded
}
# --- Legacy name: the old homepage name redirects straight onto www ---
@@ -100,9 +113,7 @@ auth.franksdank.cloud {
# --- SearXNG (quadlet; behind the Gluetun tunnel, published 8081) ---
search.franksdank.cloud {
reverse_proxy anubis:8923 {
header_up X-Real-Ip {remote_host}
}
import guarded
}
# --- OpenWebUI (quadlet on slirp4netns; published 3005) ---
@@ -126,30 +137,22 @@ gitea.franksdank.cloud {
# never reach Grafana — RP mode needs none and works regardless of topology.
# The basicauth edge this replaces (CADDY_BASICAUTH_HASH) is retired.
grafana.franksdank.cloud {
reverse_proxy anubis:8923 {
header_up X-Real-Ip {remote_host}
}
import guarded
}
# --- Prometheus ---
prometheus.franksdank.cloud {
reverse_proxy anubis:8923 {
header_up X-Real-Ip {remote_host}
}
import guarded
}
# --- Loki (guarded: Anubis → Authentik; loki-proxy provider, internal_host caddy:8017) ---
loki.franksdank.cloud {
reverse_proxy anubis:8923 {
header_up X-Real-Ip {remote_host}
}
import guarded
}
# --- Seerr (guarded: Anubis → Authentik; seerr provider, internal_host caddy:8017) ---
seerr.franksdank.cloud {
reverse_proxy anubis:8923 {
header_up X-Real-Ip {remote_host}
}
import guarded
}
# --- Marreta (guarded: Anubis → Authentik; marreta provider, internal_host caddy:8017) ---
@@ -157,9 +160,7 @@ seerr.franksdank.cloud {
# group gate). Egress rides the searxng gluetun tunnel; reached via the
# host-published port 8082 (gluetun netns), not by container name.
marreta.franksdank.cloud {
reverse_proxy anubis:8923 {
header_up X-Real-Ip {remote_host}
}
import guarded
}
# --- Pi-hole (quadlet on slirp4netns; published 8083) ---
@@ -233,16 +234,12 @@ books.franksdank.cloud {
# --- RomM ---
romm.franksdank.cloud {
reverse_proxy anubis:8923 {
header_up X-Real-Ip {remote_host}
}
import guarded
}
# --- Copyparty ---
files.franksdank.cloud {
reverse_proxy anubis:8923 {
header_up X-Real-Ip {remote_host}
}
import guarded
}
# --- Jellyfin ---
@@ -1,30 +1,32 @@
# Quadlet unit: node-exporter (rootless, boot-persistent).
# Quadlet unit: node-exporter (rootless, boot-persistent) — RENDERED ARTIFACT.
# Regenerate: python3 scripts/service-map.py render node-exporter-unit --host dankclaw
# Source: shared/services.yml (entry node-exporter) + this template.
#
# Install path on dankclaw: ~/.config/containers/systemd/node-exporter.container
# Unit name: node-exporter.service (container name: node-exporter)
#
# Host metrics source for Prometheus. This replaces the compose service of the
# same name in dankclaw/compose.yaml ("Observability" block).
# Host metrics source for Prometheus. Replaces the compose service of the same
# name (the "Observability" block): Dankclaw moved to this quadlet on
# 2026-09-11, Dankserver on 2026-09-16 — its compose-only copy died at a reboot
# and stayed dead (rootless Podman does not restart bare containers after the
# explicit stop shutdown performs), dead for a day and a half before anyone
# noticed. One owner per container name; no compose file remains anywhere.
#
# HOST NAMESPACE. It must run in the host network namespace to report the
# host's own interfaces/connections, and --pid=host for the processes and
# cgroups collectors. "localhost" inside the container is therefore the host's
# loopback, which is what the homepage status timer probes.
#
# LISTEN ADDRESSES. Until 2026-09-11 this bound :9100 on every interface. It
# now binds exactly two: loopback for the homepage status probe, and the LAN
# address because Prometheus — which runs in a container on dankclaw_default —
# scrapes http://192.168.0.143:9100 (shared/prometheus/prometheus.yml,
# job "node-dankclaw"). node_exporter itself has no authentication, so keeping
# the wildcard bind off the table matters: this is the second half of the
# pairing that removes the unauthenticated observability surface, and both
# consumers are accounted for. Dankserver runs its own copy of this container
# (scraped by the "node-dankserver" job) — since 2026-09-16 as a Quadlet unit in
# the same shape (dankserver/observability/quadlet/), because the compose-only
# version it replaced had been dead for a day and a half before anyone noticed.
# LISTEN ADDRESSES. Exactly two: loopback for the homepage status probe, and
# the LAN address, because Prometheus — which runs in a container on
# dankclaw_default — scrapes http://192.168.0.143:9100/metrics
# (shared/prometheus/prometheus.yml, job "node-dankclaw"). node_exporter has no
# authentication, so the wildcard bind stays off the table; this is the
# pairing that removes the unauthenticated observability surface on every
# host, and both consumers are accounted for.
#
# MOUNTS. The container reads the host's /proc, /sys and / as /host/* read-only
# and is pointed at them with --path.*; it writes nothing anywhere.
# MOUNTS. The container reads the host's /proc, /sys and / as /host/*
# read-only and is pointed at them with --path.*; it writes nothing anywhere.
[Unit]
Description=Prometheus node exporter (Dankclaw host metrics)
+30 -34
View File
@@ -1,47 +1,43 @@
# A records for the flat franksdank.name scheme (dnsmasq addn-hosts format:
# <IP> <name> [<name> ...]). Served by the Pi-hole quadlet (FTL is a
# dnsmasq fork): the quadlet mounts this file into /etc/pihole/hosts/ read-only.
# A records for the flat franksdank.cloud scheme (dnsmasq addn-hosts
# format: <IP> <name>). RENDERED — do not edit by hand:
#
# Caddy is consolidated on dankclaw (192.168.0.143): it fronts every service
# on both hosts, so every name resolves to that one IP. Keep in sync with the
# Caddyfile (tests/pihole.bats cross-checks this).
# scripts/service-map.py render pihole-hosts > dankclaw/pihole/addn-hosts
#
# 192.168.0.143 = dankclaw (runs the Caddy container + the AI/dev stack)
# 192.168.0.235 = dankserver (media stack; reached by Caddy via published
# LAN ports — no DNS records point here)
# The source of truth is the `dns:` field of shared/services.yml; the
# drift gate in tests/service_inventory.bats fails when this file and
# the inventory disagree. Caddy is consolidated on Dankclaw, so every
# name — including the services running on dankserver and
# frankbook-pro — resolves at the Dankclaw edge.
#
# dnsmasq strips ONLY whole-line comments in addn-hosts: a trailing
# comment on a data line becomes a HOSTNAME token. No inline notes
# here; the service attribution lives in the inventory.
# --- Homepage (naked domain + www) + legacy names ---
192.168.0.143 search.franksdank.cloud
192.168.0.143 marreta.franksdank.cloud
192.168.0.143 pihole.franksdank.cloud
192.168.0.143 ai.franksdank.cloud
192.168.0.143 gitea.franksdank.cloud
192.168.0.143 comfyui.franksdank.cloud
192.168.0.143 seerr.franksdank.cloud
192.168.0.143 hermes.franksdank.cloud
192.168.0.143 hermesadmin.franksdank.cloud
192.168.0.143 prometheus.franksdank.cloud
192.168.0.143 grafana.franksdank.cloud
192.168.0.143 loki.franksdank.cloud
192.168.0.143 auth.franksdank.cloud
192.168.0.143 franksdank.cloud
192.168.0.143 www.franksdank.cloud
192.168.0.143 dankclaw.franksdank.cloud
192.168.0.143 dankserver.franksdank.cloud
# --- Authentik (SSO / OIDC provider) ---
192.168.0.143 auth.franksdank.cloud
# --- Media services (dankserver, proxied) ---
192.168.0.143 plex.franksdank.cloud
192.168.0.143 plex.franksdank.casa
192.168.0.143 books.franksdank.cloud
192.168.0.143 romm.franksdank.cloud
192.168.0.143 files.franksdank.cloud
192.168.0.143 jelly.franksdank.cloud
192.168.0.143 books.franksdank.cloud
192.168.0.143 plex.franksdank.cloud
192.168.0.143 plex.franksdank.casa
192.168.0.143 prowlarr.franksdank.cloud
192.168.0.143 qbittorrent.franksdank.cloud
192.168.0.143 radarr.franksdank.cloud
192.168.0.143 lidarr.franksdank.cloud
192.168.0.143 readarr.franksdank.cloud
192.168.0.143 prowlarr.franksdank.cloud
192.168.0.143 qbittorrent.franksdank.cloud
# --- AI / search / dev services (dankclaw, local) ---
192.168.0.143 search.franksdank.cloud
192.168.0.143 comfyui.franksdank.cloud
192.168.0.143 ai.franksdank.cloud
192.168.0.143 hermes.franksdank.cloud
192.168.0.143 hermesadmin.franksdank.cloud
192.168.0.143 gitea.franksdank.cloud
192.168.0.143 grafana.franksdank.cloud
192.168.0.143 prometheus.franksdank.cloud
192.168.0.143 loki.franksdank.cloud
192.168.0.143 pihole.franksdank.cloud
192.168.0.143 seerr.franksdank.cloud
192.168.0.143 marreta.franksdank.cloud
@@ -1,34 +1,32 @@
# Quadlet unit: node-exporter (rootless, boot-persistent) — DANKSERVER.
# Quadlet unit: node-exporter (rootless, boot-persistent) — RENDERED ARTIFACT.
# Regenerate: python3 scripts/service-map.py render node-exporter-unit --host dankserver
# Source: shared/services.yml (entry node-exporter) + this template.
#
# Install path on dankserver: ~/.config/containers/systemd/node-exporter.container
# Unit name: node-exporter.service (container name: node-exporter)
#
# WHY THIS FILE EXISTS. Until 2026-09-16 the Dankserver node-exporter ran only
# from `dankserver/compose.yaml` (the "Observability" block, `restart:
# unless-stopped`). Rootless Podman does not restart bare containers after the
# explicit stop that shutdown performs, so it died at a reboot and stayed dead:
# `podman ps -a` showed it `Exited (143)` and Prometheus' `node-dankserver` job
# had been scraping nothing since. The compose entry also bound the default
# wildcard address, which is the unauthenticated-metrics-API-on-every-interface
# posture that was already removed on Dankclaw. This unit replaces both: one
# owner per container name (the compose service is deleted), explicit listen
# addresses, and a generator-provided boot path.
# Host metrics source for Prometheus. Replaces the compose service of the same
# name (the "Observability" block): Dankclaw moved to this quadlet on
# 2026-09-11, Dankserver on 2026-09-16 — its compose-only copy died at a reboot
# and stayed dead (rootless Podman does not restart bare containers after the
# explicit stop shutdown performs), dead for a day and a half before anyone
# noticed. One owner per container name; no compose file remains anywhere.
#
# HOST NAMESPACE. Networking and PID in the host's namespaces: the collectors
# report the host's own interfaces, connections, processes and cgroups. The
# host's /proc, /sys and / are mounted read-only under /host and /rootfs, and
# node_exporter writes nothing.
# HOST NAMESPACE. It must run in the host network namespace to report the
# host's own interfaces/connections, and --pid=host for the processes and
# cgroups collectors. "localhost" inside the container is therefore the host's
# loopback, which is what the homepage status timer probes.
#
# LISTEN ADDRESSES. Exactly two:
# - 127.0.0.1:9100 for host-local probes
# - 192.168.0.235:9100 for Prometheus, which scrapes this host from Dankclaw
# (shared/prometheus/prometheus.yml, job "node-dankserver"). It cannot use
# loopback across hosts, and node_exporter has no authentication, so the
# wildcard bind stays off the table. This is the same deliberate LAN
# listener posture the observability runbook documents.
# LISTEN ADDRESSES. Exactly two: loopback for the homepage status probe, and
# the LAN address, because Prometheus — which runs in a container on
# dankclaw_default — scrapes http://192.168.0.235:9100/metrics
# (shared/prometheus/prometheus.yml, job "node-dankserver"). node_exporter has no
# authentication, so the wildcard bind stays off the table; this is the
# pairing that removes the unauthenticated observability surface on every
# host, and both consumers are accounted for.
#
# MIRROR. Keep this file in step with dankclaw/observability/quadlet/
# node-exporter.container; only the Description and the LAN address differ.
# MOUNTS. The container reads the host's /proc, /sys and / as /host/*
# read-only and is pointed at them with --path.*; it writes nothing anywhere.
[Unit]
Description=Prometheus node exporter (Dankserver host metrics)
+2 -2
View File
@@ -26,8 +26,8 @@ RECIPE=/home/dank/serverAppConfigs-swift15/scripts/recipes/qwen4exp-swift15-w4dn
CALIB=/home/dank/corpora/calib
OUT=/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad
LOG=/home/dank/logs/swift15_convert.log
# gate_up ~37.6 + down ~18.8 + PLE 47.7 + trunk/embeddings/head ~10 GiB (all-w4 expert table;
# the plugin's one-encoding law dropped the block-fp8 down tier -- see the recipe header).
# gate_up ~37.6 + down ~18.8 + PLE 47.7 + trunk/embeddings/head ~10 GiB (all-w4 expert
# table; one encoding per expert table rules out a block-fp8 down tier -- see the recipe header).
MIN_MIB=${FRANK_SWIFT15_RAD_MIN_MIB:-108000}
# Gate 1: inputs. A missing Gram directory converts UNCALIBRATED with a
+54 -461
View File
@@ -1,475 +1,68 @@
#!/usr/bin/env python3
"""Plan runtime services affected by repository changes.
"""deploy-plan.py — map changed paths to deployable services.
The planner is intentionally side-effect free. It accepts either an explicit
path list or two Git revisions. The latter lets compose-file changes resolve
to only the service definitions that actually changed instead of restarting an
entire compose project.
Thin wrapper over scripts/service-map.py, which owns the path->service
semantics in the shared inventory (shared/services.yml): per-service path
rules, the quadlet netns/network coupling graph, host scoping (a host's
directory can only select that host's services), and canonical restart
order. This file keeps the deployer's CLI contract only.
Usage:
deploy-plan.py --target NAME --repo DIR --base REV --head REV
deploy-plan.py --target NAME [paths...] (paths also on stdin)
Prints one service name per line (canonical restart order); exit 0 with no
output when nothing relevant changed.
"""
from __future__ import annotations
import argparse
import os
import subprocess
import sys
from pathlib import PurePosixPath
from typing import Any
import yaml
# Dependency-sensitive services come before their consumers. Independent
# compose services follow. LLM services are deliberately last and are never
# implied by a generic host, stack, documentation, or config change.
SERVICE_ORDER = (
"gluetun",
"searxng-valkey",
"searxng",
"marreta",
"pihole",
"open-webui",
"gitea-postgres",
"gitea-server",
"comfyui",
"seerr",
"hermes",
"hermes-dashboard",
"hermes-webui",
"prometheus",
"grafana",
"loki",
"alloy",
"node-exporter",
"alertmanager",
"authentik-postgres",
"authentik-server",
"authentik-worker",
"authentik-outpost",
"authentik-outpost-ldap",
"caddy",
"homepage",
"romm-mariadb",
"romm-redis",
"romm",
"copyparty",
"jellyfin",
"calibre-web",
"calibre-server",
"plex",
"tautulli",
"prowlarr",
"qbittorrent",
"radarr",
"sonarr",
"lidarr",
"readarr",
"frank-dragoon",
"frank-zealot",
"frank-archon",
)
DANKCLAW_SERVICES = {
"hermes-webui",
"gluetun",
"searxng-valkey",
"searxng",
"marreta",
"pihole",
"open-webui",
"gitea-postgres",
"gitea-server",
"comfyui",
"seerr",
"hermes",
"hermes-dashboard",
"prometheus",
"grafana",
"loki",
"alloy",
"node-exporter",
"alertmanager",
"authentik-postgres",
"authentik-server",
"authentik-worker",
"authentik-outpost",
"authentik-outpost-ldap",
"caddy",
"homepage",
"frank-dragoon",
"frank-zealot",
}
DANKSERVER_SERVICES = {
"romm-mariadb",
"romm-redis",
"romm",
"copyparty",
"jellyfin",
"calibre-web",
"calibre-server",
"plex",
"tautulli",
"prowlarr",
"qbittorrent",
"radarr",
"sonarr",
"lidarr",
"readarr",
"frank-archon",
"node-exporter",
}
# FrankBook-Pro runs the *arr stack + qBittorrent inside a podman machine on
# macOS. Service names overlap the Linux hosts (gluetun, qbittorrent, ...);
# the per-target sets and the frankbook-pro intercept in deploy-remote.sh
# keep dispatch unambiguous.
FRANKBOOK_SERVICES = {
"gluetun",
"qbittorrent",
"prowlarr",
"radarr",
"sonarr",
"lidarr",
"readarr",
}
DANKCLAW_COMPOSE_SERVICES = set()
# Dankserver's compose file is still the source of truth for its media stack;
# Dankclaw's is gone (deleted 2026-09-16). Every service on Dankclaw is a
# Quadlet unit now — the last compose services were the hermes trio, whose units
# have owned their containers since 2026-09-12 and whose compose blocks were
# dead weight that could only create a duplicate container.
DANKSERVER_COMPOSE_SERVICES = {
"mariadb",
"redis",
"romm",
"copyparty",
"jellyfin",
"calibre-web",
}
COMPOSE_NAME_MAP = {"mariadb": "romm-mariadb", "redis": "romm-redis"}
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
SERVICE_MAP = os.path.join(SCRIPT_DIR, "service-map.py")
def _add(services: set[str], *names: str) -> None:
services.update(names)
def target_services(target: str) -> set[str]:
if target == "dankclaw":
return DANKCLAW_SERVICES
if target == "dankserver":
return DANKSERVER_SERVICES
if target == "frankbook-pro":
return FRANKBOOK_SERVICES
raise ValueError(f"unknown target: {target}")
def _compose_services(target: str) -> set[str]:
return DANKCLAW_COMPOSE_SERVICES if target == "dankclaw" else DANKSERVER_COMPOSE_SERVICES
def _git_show(repo: str, revision: str, path: str) -> str | None:
result = subprocess.run(
["git", "-C", repo, "show", f"{revision}:{path}"],
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
check=False,
)
return result.stdout if result.returncode == 0 else None
def _yaml_document(contents: str | None) -> dict[str, Any]:
if contents is None:
return {}
try:
data = yaml.safe_load(contents) or {}
except yaml.YAMLError:
return {}
return data if isinstance(data, dict) else {}
def _yaml_services(contents: str | None) -> dict[str, Any]:
services = _yaml_document(contents).get("services", {})
return services if isinstance(services, dict) else {}
def compose_changes(repo: str, target: str, base: str, head: str) -> set[str]:
"""Return only compose services whose YAML definitions or dependencies changed.
A target with no compose file (Dankclaw since 2026-09-16) plans nothing:
its services are Quadlet units, selected by their own paths.
"""
path = f"{target}/compose.yaml"
if not _git_show(repo, head, path) and not _git_show(repo, base, path):
return set()
old_doc = _yaml_document(_git_show(repo, base, path))
new_doc = _yaml_document(_git_show(repo, head, path))
old = old_doc.get("services", {}) if isinstance(old_doc.get("services", {}), dict) else {}
new = new_doc.get("services", {}) if isinstance(new_doc.get("services", {}), dict) else {}
names = set(old) | set(new)
allowed = _compose_services(target)
# An unavailable or malformed revision is safer as a broad compose plan
# than as a silent no-op. It still cannot include LLM units.
if (old_doc and not new_doc) or (new_doc and not old_doc and names):
return {COMPOSE_NAME_MAP.get(n, n) for n in allowed}
# Top-level networks/configuration can affect every compose service even
# when no individual service mapping changed. Ignore the services key while
# comparing the rest of the Compose document.
old_globals = {key: value for key, value in old_doc.items() if key != "services"}
new_globals = {key: value for key, value in new_doc.items() if key != "services"}
if old_globals != new_globals:
return {COMPOSE_NAME_MAP.get(n, n) for n in allowed}
changed = {name for name in names if name in allowed and old.get(name) != new.get(name)}
return {COMPOSE_NAME_MAP.get(n, n) for n in changed}
def services_for_path(path: str, target: str) -> set[str]:
"""Return services affected by one repository-relative path."""
path = path.replace("\\", "/")
p = PurePosixPath(path)
parts = p.parts
services: set[str] = set()
if not parts:
return services
# Non-runtime material is deliberately a no-op.
if parts[0] in {"docs", "tests", "old", ".githooks"}:
return services
if path in {"README.md", "AGENTS.md"} or parts[0] == ".obsidian":
return services
if parts[0] == "shared":
if len(parts) < 2:
return services
group = parts[1]
if group == "searxng":
_add(services, "searxng")
elif group == "prometheus":
_add(services, "prometheus")
if len(parts) >= 3 and parts[2] == "alertmanager.yml":
# The Alertmanager config ships from this directory: a change
# to it must redeploy the service that installs it.
_add(services, "alertmanager")
elif group == "loki":
_add(services, "loki")
elif group == "alloy":
_add(services, "alloy")
elif group == "grafana":
# Provisioned datasources and dashboards: git is the source of
# truth, so a change here must restart the service that reads them.
_add(services, "grafana")
elif group == "homepage":
# Homepage is a Dankclaw-owned deployment. Shared homepage files
# are intentionally not copied during a Dankserver-only run.
if target == "dankclaw":
_add(services, "homepage")
return services
if parts[0] == "dankclaw":
if len(parts) >= 2 and parts[1] == "compose.yaml":
# Dankclaw has no compose file since 2026-09-16: every service is a
# Quadlet unit, each selected by its own path. A change (or a
# deletion) of this path therefore restarts nothing.
return services
if len(parts) >= 2 and parts[1] == "Caddyfile":
_add(services, "caddy")
return services
if len(parts) >= 2 and parts[1] == "Containerfile.comfyui":
_add(services, "comfyui")
return services
if len(parts) >= 2 and parts[1] in {"caddy", "comfyui", "loki", "alloy", "hermes", "anubis", "seerr"}:
_add(services, parts[1])
return services
if len(parts) >= 4 and parts[1] == "authentik" and parts[2] == "quadlet":
name = parts[3].removesuffix(".container")
if name in {"authentik-postgres", "authentik-server", "authentik-worker", "authentik-outpost", "authentik-outpost-ldap"}:
_add(services, name)
return services
if len(parts) >= 2 and parts[1] == "searxng":
if len(parts) >= 4 and parts[2] == "quadlet":
name = parts[3]
if name == "gluetun.container":
# SearXNG and Marreta (+ its browser companion) share
# Gluetun's network namespace. A Gluetun recreation
# tears the netns down, so every sharer must restart.
_add(services, "gluetun", "searxng", "marreta")
elif name == "searxng.container":
_add(services, "searxng")
elif name == "searxng-valkey.container":
_add(services, "searxng-valkey", "searxng")
elif name == "searxng.network":
_add(services, "gluetun", "searxng-valkey", "searxng", "marreta")
return services
if len(parts) >= 2 and parts[1] == "marreta":
# One deployable owns both the app and its browser companion;
# both live in gluetun's netns. A change to either unit selects
# the marreta service, which restarts browser-then-app.
_add(services, "marreta")
return services
if len(parts) >= 2 and parts[1] == "pihole":
_add(services, "pihole")
return services
if len(parts) >= 2 and parts[1] == "open-webui":
_add(services, "open-webui")
return services
# frank-dragoon left the quadlet shape on 2026-09-19 (CIRU v4.4.1 host
# build, root unit): its files live in scripts/ and map there.
if len(parts) >= 2 and parts[1] == "observability":
# The five observability containers became Quadlet units on
# 2026-09-11. A unit change selects exactly its own service;
# SERVICE_ORDER already puts each dependency ahead of its
# consumer (prometheus before grafana, loki before alloy).
if len(parts) >= 4 and parts[2] == "quadlet":
name = parts[3].removesuffix(".container")
if name in {"prometheus", "grafana", "loki", "alloy", "node-exporter", "alertmanager"}:
_add(services, name)
return services
if len(parts) >= 2 and parts[1] == "gitea":
if len(parts) >= 4 and parts[2] == "quadlet":
name = parts[3]
if name == "gitea-postgres.container":
_add(services, "gitea-postgres")
elif name == "gitea-server.container":
_add(services, "gitea-server")
elif name == "gitea-caddy.container":
# Removing the old split proxy requires the canonical
# Caddy handler to retire its installed unit/container
# before it claims ports 80/443.
_add(services, "caddy")
elif name == "gitea.network":
_add(services, "gitea-postgres", "gitea-server")
return services
return services
if parts[0] == "dankserver":
if len(parts) >= 2 and parts[1] == "compose.yaml":
# Compose service keys map onto the deployed unit names.
_add(services, *(COMPOSE_NAME_MAP.get(n, n) for n in DANKSERVER_COMPOSE_SERVICES))
return services
if len(parts) >= 3 and parts[2] == "quadlet":
quadlet_services = {
"romm": {"romm", "romm-mariadb", "romm-redis"},
"jellyfin": {"jellyfin"},
"calibre-web": {"calibre-web"},
"calibre-server": {"calibre-server"},
"copyparty": {"copyparty"},
"plex": {"plex"},
"tautulli": {"tautulli"},
}
_add(services, *quadlet_services.get(parts[1], set()))
if len(parts) >= 2 and parts[1] == "observability":
# Dankserver's node-exporter became a Quadlet unit on 2026-09-16
# (serverAppConfigs-docs/Infrastructure/Observability.md): its
# compose service died at a reboot and stayed dead, so Prometheus'
# node-dankserver job had nothing to scrape.
if len(parts) >= 4 and parts[2] == "quadlet":
name = parts[3].removesuffix(".container")
if name == "node-exporter":
_add(services, "node-exporter")
return services
# frank-archon's quadlet-era directory left the tree on 2026-09-26
# (the Lucebox DS4.1 unit was retired for the llama.cpp root unit —
# the trio in scripts/ now maps it). Nothing lives under
# dankserver/frank-archon/ except dated trash, and trash plans
# nothing (the retire rule above the service blocks).
return services
if parts[0] == "frankbook-pro":
if len(parts) >= 3 and parts[2] == "quadlet":
name = parts[3].removesuffix(".container")
if name == "gluetun":
# Every *arr + qBittorrent container shares gluetun's
# network namespace; a gluetun recreation kills all of
# them, so the plan restarts the whole netns family.
_add(services, "gluetun", "qbittorrent",
"prowlarr", "sonarr", "radarr", "lidarr", "readarr")
elif name in FRANKBOOK_SERVICES:
_add(services, name)
return services
if parts[0] == "scripts":
name = p.name
if name in {"homepage-status.py", "homepage-status.service", "homepage-status.timer"}:
_add(services, "homepage")
if name in {"signal-relay.py", "signal-relay.service"}:
# The Signal relay is installed by the alertmanager deploy, so a
# change to it redeploying alertmanager is the only path that
# reaches it on the host.
_add(services, "alertmanager")
if name in {"frank-zealot.env", "frank-zealot.service", "serve-frank-zealot.sh"}:
# llama.cpp frank-zealot (2026-09-16) keeps its unit, env and serve
# files here; each one selects frank-zealot only. The ik_llama.cpp
# trio that lived here for two days is in scripts/.AGENTTRASH/,
# which is archive material and plans nothing.
_add(services, "frank-zealot")
if name in {"frank-dragoon.env", "frank-dragoon.service", "serve-frank-dragoon.sh"}:
# CIRU v4.4.1 frank-dragoon (2026-09-19) keeps its unit, env and
# serve files here the same way: a root unit on a host-built
# runtime. The download script fetches the payload, which is
# host-side state, so it selects nothing.
_add(services, "frank-dragoon")
if name == "qwen-sharp-v22.4.0.jinja":
# The Qwen-Sharp chat template frank-dragoon loads via
# --chat-template-file. It lived under dankserver/frank-medium/ until
# the 2026-09-26 cutover retired that directory; editing it changes
# what the live frank-dragoon endpoint renders, so it plans
# frank-dragoon.
_add(services, "frank-dragoon")
if name in {"frank-archon.env", "frank-archon.service", "serve-frank-archon.sh"}:
# llama.cpp frank-archon (2026-09-26, replacing the Lucebox
# DS4.1 quadlet) keeps its unit, env and serve files here the
# same way: a root unit on the unsloth b11030-mix prebuilt
# serving Qwen3.8-Flash-Next Q8_0. The download script fetches
# the payload, which is host-side state, so it selects nothing.
_add(services, "frank-archon")
# Deployment/test/benchmark scripts and other repo tooling do not
# restart a runtime service.
return services
# Examples and env templates are not deployed runtime state.
if p.name in {".env.example", "env.example"}:
return services
return services
def changed_paths(repo: str, base: str, head: str) -> list[str]:
result = subprocess.run(
["git", "-C", repo, "diff", "--name-only", "--diff-filter=ACDMRTUXB", base, head],
text=True,
stdout=subprocess.PIPE,
check=True,
)
return [line for line in result.stdout.splitlines() if line]
def plan(paths: list[str], target: str, repo: str | None = None,
base: str | None = None, head: str | None = None) -> list[str]:
services: set[str] = set()
for path in paths:
if path == f"{target}/compose.yaml" and repo and base and head:
services.update(compose_changes(repo, target, base, head))
else:
services.update(services_for_path(path, target))
allowed = target_services(target)
return [name for name in SERVICE_ORDER if name in services and name in allowed]
def git_paths(repo: str, base: str, head: str) -> list[str]:
proc = subprocess.run(
["git", "-C", repo, "diff", "--name-only", base, head],
text=True, capture_output=True)
if proc.returncode != 0:
sys.stderr.write(f"deploy-plan: git diff failed: {proc.stderr}")
raise SystemExit(1)
return [line for line in proc.stdout.splitlines() if line]
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--target", choices=("dankclaw", "dankserver", "frankbook-pro"), required=True)
parser.add_argument("--paths", nargs="+")
parser.add_argument("--repo", help="repository path for revision-aware planning")
parser.add_argument("--base", help="base Git revision")
parser.add_argument("--head", help="head Git revision")
args = parser.parse_args()
if not args.paths and not (args.repo and args.base and args.head):
parser.error("provide --paths or all of --repo, --base, and --head")
paths = args.paths or changed_paths(args.repo, args.base, args.head)
print("\n".join(plan(paths, args.target, args.repo, args.base, args.head)))
args = sys.argv[1:]
if not args or args[0] != "--target":
print(__doc__, file=sys.stderr)
return 2
if len(args) < 2:
print("missing target name after --target", file=sys.stderr)
return 2
target = args[1]
# `--paths` is the historic seam keyword: a flag, not an option — the
# paths are the positional arguments that follow it.
rest = [a for a in args[2:] if a != "--paths"]
if "--repo" in rest:
try:
repo = rest[rest.index("--repo") + 1]
base = rest[rest.index("--base") + 1]
head = rest[rest.index("--head") + 1]
except (ValueError, IndexError):
print("--repo, --base and --head must appear together", file=sys.stderr)
return 2
paths = git_paths(repo, base, head)
else:
paths = rest or [line.strip() for line in sys.stdin if line.strip()]
if not paths:
return 0
proc = subprocess.run([sys.executable, SERVICE_MAP, "plan", "--host", target] + paths,
text=True, capture_output=True)
if proc.returncode != 0:
sys.stderr.write(proc.stderr)
return proc.returncode
sys.stdout.write(proc.stdout)
return 0
+163 -233
View File
@@ -9,6 +9,13 @@
# pending until its service is explicitly deployed.
set -euo pipefail
# The LLM handlers' artifact gates are the serve scripts' own library, so
# the deployer and the unit refuse the same missing payload with the same
# words — no deploy can pass a gate the unit would fail. GATE_PREFIX is set
# per handler.
# shellcheck disable=SC1091
source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/llm-gates.sh"
TARGET="${1:-}"
REVISION="${2:-}"
shift 2 || true
@@ -80,31 +87,10 @@ cd "$REMOTE_DIR"
mkdir -p "$STATE_DIR"
service_is_allowed() {
local service="$1"
case "$TARGET:$service" in
dankclaw:gluetun|dankclaw:searxng-valkey|dankclaw:searxng|dankclaw:pihole|\
dankclaw:open-webui|dankclaw:gitea-postgres|dankclaw:gitea-server|\
dankclaw:comfyui|dankclaw:hermes|\
dankclaw:hermes-dashboard|dankclaw:hermes-webui|dankclaw:prometheus|dankclaw:grafana|\
dankclaw:loki|dankclaw:alloy|dankclaw:node-exporter|dankclaw:alertmanager|\
dankclaw:authentik-postgres|dankclaw:authentik-server|\
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|\
dankclaw:caddy|dankclaw:homepage|dankclaw:seerr|dankclaw:marreta|\
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
dankserver:prowlarr|dankserver:qbittorrent|dankserver:radarr|\
dankserver:sonarr|dankserver:lidarr|dankserver:readarr|\
dankserver:frank-archon|dankserver:node-exporter|\
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
frankbook-pro:readarr)
return 0
;;
*)
return 1
;;
esac
# The inventory (shared/services.yml via scripts/service-map.py) is the
# only allow-list: a service is deployable on a target iff the inventory
# places it there. Retiring a service from a host is a data edit.
python3 scripts/service-map.py allows --host "$TARGET" --service "$1"
}
mark_status() {
@@ -274,13 +260,11 @@ install_signal_relay() {
}
copy_loki_config() {
run_privileged mkdir -p /opt/loki/config
run_privileged cp shared/loki/loki-config.yaml /opt/loki/config/loki-config.yaml
copy_config /opt/loki/config shared/loki/loki-config.yaml
}
copy_alloy_config() {
run_privileged mkdir -p /opt/alloy/config
run_privileged cp shared/alloy/config.alloy /opt/alloy/config/config.alloy
copy_config /opt/alloy/config shared/alloy/config.alloy
}
copy_grafana_provisioning() {
@@ -289,14 +273,11 @@ copy_grafana_provisioning() {
# datasource would be invisible to review, lost with the data volume, and
# unknown to anyone reading the repository. The unit already mounts
# /opt/grafana/provisioning read-only, so this only has to populate it.
run_privileged mkdir -p /opt/grafana/provisioning/datasources \
/opt/grafana/provisioning/dashboards
run_privileged cp shared/grafana/provisioning/datasources/prometheus.yml \
/opt/grafana/provisioning/datasources/prometheus.yml
run_privileged cp shared/grafana/provisioning/dashboards/provider.yml \
/opt/grafana/provisioning/dashboards/provider.yml
run_privileged cp shared/grafana/provisioning/dashboards/llm-fleet.json \
/opt/grafana/provisioning/dashboards/llm-fleet.json
copy_config /opt/grafana/provisioning/datasources \
shared/grafana/provisioning/datasources/prometheus.yml
copy_config /opt/grafana/provisioning/dashboards \
shared/grafana/provisioning/dashboards/provider.yml \
shared/grafana/provisioning/dashboards/llm-fleet.json
}
render_caddyfile() {
@@ -441,31 +422,6 @@ deploy_caddy() {
fi
}
deploy_compose_service() {
local service="$1"
if [[ "$TARGET" == dankserver ]]; then
# Dankserver's compose file is the source-of-truth inventory, but its
# production containers are individually systemd-managed/bare. Do not
# use compose up here: it can create a second container with a wrong
# name and can interfere with a running workload.
echo "WARN: $service is recorded in dankserver/compose.yaml; production container management is host-specific and no restart was attempted."
return 0
fi
if [[ "$TARGET" == dankclaw ]]; then
# Dankclaw has no compose file since 2026-09-16: every service is a
# Quadlet unit with its own handler above. Reaching this means the
# dispatcher is missing a case, and the old fallback (`podman-compose
# up -d`) would now be a silent no-op or a duplicate container.
echo "no compose services on dankclaw; $service has no quadlet handler" >&2
return 2
fi
case "$service" in
*)
echo "unknown compose service: $service" >&2
return 2
;;
esac
}
# grafana is the one observability service that holds a credential: the
# Authentik OAuth2 client secret. It is seeded into a per-service env file (the
@@ -568,6 +524,35 @@ quadlet_unit_dir="$HOME/.config/containers/systemd"
# dankserver's user manager had no systemd container generator dir before the
# quadlet migration; every handler that installs a unit must find it present.
mkdir -p "$quadlet_unit_dir"
# copy_config DEST SRC... — the privileged config copy every service uses:
# ensure DEST, copy each tracked SRC into it. The deployer owns /opt state;
# the files in git are the source of truth, so a service config that only
# exists on the host is a gap by policy.
copy_config() {
local dest="$1"; shift
run_privileged mkdir -p "$dest"
local src
for src in "$@"; do
run_privileged cp "$src" "$dest/"
done
}
# retire_unit_file PATH TRASH [UNIT] — the repo Deletion & Trash policy, in
# code: stop and disable the named unit, move the unit FILE into the home
# trash (never rm), report. The dragoon quadlet retirement and the
# compose-era comfyui.service retirement were the same six lines twice.
retire_unit_file() {
local path="$1" trash="$2" unit="${3:-}"
[[ -e "$path" ]] || return 0
if [[ -n "$unit" ]]; then
systemctl --user stop "$unit" 2>/dev/null || true
systemctl --user disable "$unit" 2>/dev/null || true
fi
mkdir -p "$trash"
mv -- "$path" "$trash/$(basename "$path")"
echo "retired $(basename "$path") -> $trash"
}
install_quadlet() {
local service="$1"
local source_dir="$2"
@@ -655,15 +640,8 @@ deploy_comfyui() {
# generator produces, and it is what kept a stale, device-less container
# alive through `restart: unless-stopped` (9,146 failures on
# "No HIP GPUs are available"). Never `rm`: it moves into the home trash.
local legacy="$HOME/.config/systemd/user/comfyui.service"
if [[ -e "$legacy" ]]; then
local trash="$HOME/.AGENTTRASH/20260916-comfyui-compose-unit"
mkdir -p "$trash"
systemctl --user stop comfyui.service 2>/dev/null || true
systemctl --user disable comfyui.service 2>/dev/null || true
mv -- "$legacy" "$trash/comfyui.service"
echo "retired the compose-era comfyui.service -> $trash"
fi
retire_unit_file "$HOME/.config/systemd/user/comfyui.service" \
"$HOME/.AGENTTRASH/20260916-comfyui-compose-unit" comfyui.service
# The stale container carries the old config (no /dev/kfd, no /dev/dri) and
# would keep the quadlet's container from taking the name. It is replaced
# by the unit in the next step, so nothing is lost.
@@ -721,9 +699,15 @@ deploy_gitea_unit() {
systemctl --user restart "$service.service"
}
# The root units are symlinks from the systemd system directory into the
# checkout. SYSTEMD_SYSTEM_DIR is production (/etc/systemd/system); the knob
# exists so the deploy seam can run a whole handler — relink included — in a
# sandbox and read the unit links as evidence, unprivileged.
SYSTEMD_SYSTEM_DIR="${SYSTEMD_SYSTEM_DIR:-/etc/systemd/system}"
relink_unit() {
local service="$1"
run_privileged ln -sfn "$REMOTE_DIR/scripts/$service.service" "/etc/systemd/system/$service.service"
run_privileged ln -sfn "$REMOTE_DIR/scripts/$service.service" "$SYSTEMD_SYSTEM_DIR/$service.service"
}
# Authentik runs as three boot-persistent Quadlet units (postgres -> server ->
@@ -942,17 +926,17 @@ deploy_plex() {
systemctl --user restart plex.service
# Readiness gate: /identity answers MediaContainer XML once PMS is up.
#
# HTTPS, NOT HTTP. PMS drops plain HTTP off-loopback: the probe's
# connection arrives on the container's eth0, so an http:// request is
# accepted and then answered with nothing (curl 52, Empty reply) against a
# perfectly healthy server. Measured 2026-10-02 — the gate burned its full
# 600s and failed a plex whose https://127.0.0.1:32400/identity answered
# 200 in 44 ms the whole time. The same law is why the Caddyfile dials the
# plex upstream over TLS (tests/caddyfile.bats pins it).
# HTTPS, NOT HTTP. PMS serves plain HTTP on loopback only: a probe
# arriving on the container's interface gets its connection accepted and
# then an empty reply (curl 52) from a perfectly healthy server, so an
# http:// probe would burn the whole timeout and fail a service that
# answers https://127.0.0.1:32400/identity in milliseconds. The Caddyfile
# dials the same upstream over TLS for the same reason
# (tests/caddyfile.bats pins it).
# -k: PMS presents its own self-signed cert on 32400.
#
# Reality check from the 2026-09 cutover: the FIRST container boot of a
# years-old library ran ~10 min, so the gate still allows 600s.
# The budget accounts for a first boot of a large pre-existing library,
# which spends minutes indexing before it answers anything.
# Fail the deploy (pending state preserved) rather than mark it deployed blind.
for ((i = 1; i <= 120; i++)); do
curl -fsSk --max-time 3 https://127.0.0.1:32400/identity >/dev/null && return 0
@@ -1125,13 +1109,8 @@ LLM_DRAGOON_TOK_DIR="${LLM_DRAGOON_TOK_DIR:-/home/dank/models/halogen-flashnext-
LLM_DRAGOON_TOK_FILES="${LLM_DRAGOON_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json}"
retire_halogen_dragoon_quadlet() {
local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet"
systemctl --user stop frank-dragoon.service 2>/dev/null || true
if [[ -e "$quadlet_unit_dir/frank-dragoon.container" ]]; then
mkdir -p "$trash"
mv -- "$quadlet_unit_dir/frank-dragoon.container" "$trash/frank-dragoon.container"
echo "retired quadlet frank-dragoon.container -> $trash"
fi
retire_unit_file "$quadlet_unit_dir/frank-dragoon.container" \
"$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet" frank-dragoon.service
systemctl --user daemon-reload
}
@@ -1153,73 +1132,71 @@ wait_port_free() {
ss -tan "sport = :$port" >&2 || true
return 1
}
# assert_unit_owns_port SERVICE PORT — the false-green gate every LLM
# endpoint shares. A health answer proves the PORT replied, never WHO
# replied: on 2026-09-26 a leftover probe held :7770, satisfied the curl,
# and the deploy reported green over a stopped unit. Green requires the
# unit's own MainPID to be alive behind the answer.
assert_unit_owns_port() {
local service="$1" port="$2" main_pid
main_pid="$(run_privileged systemctl show -p MainPID --value "$service.service")"
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
echo "FAIL: :$port answered but $service.service has no MainPID — a stray process holds the port" >&2
return 1
fi
echo "$service: health answered by unit MainPID $main_pid"
}
# start_root_unit — the tail every LLM root unit deploys: relink the unit
# file from the checkout into /etc/systemd/system, (re)enable, restart,
# wait for /health, and prove the answer came from the unit's own process.
# The three endpoints differ in gates, ports, and budgets, never in this.
start_root_unit() {
local service="$1" port="$2" health_limit="$3"
relink_unit "$service"
run_privileged systemctl daemon-reload
run_privileged systemctl enable "$service.service"
run_privileged systemctl restart "$service.service"
wait_for_health "$service" "http://127.0.0.1:${port}/health" "$health_limit" || return 1
assert_unit_owns_port "$service" "$port"
}
deploy_llm_dragoon() {
# Refuse rather than restart-loop; every gate below has a twin in
# serve-frank-dragoon.sh and they must agree (2026-10-01 lesson).
# Capture podman output before grepping: `podman images | grep -q`
# races under pipefail (SIGPIPE 141 becomes "no match", 2026-09-30).
if ! grep -qF "${LLM_DRAGOON_IMAGE#*@}" <<<"$(podman images --format '{{.RepoDigests}}' 2>/dev/null)"; then
echo "ERROR: halogen image absent: $LLM_DRAGOON_IMAGE" >&2
echo "frank-dragoon: pull it: podman pull ${LLM_DRAGOON_IMAGE%@*}" >&2
return 1
fi
if [[ ! -f "$LLM_DRAGOON_CKPT" ]]; then
echo "ERROR: halogen checkpoint absent at $LLM_DRAGOON_CKPT" >&2
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
return 1
fi
if [[ "$(stat -c %s "$LLM_DRAGOON_CKPT")" != "$LLM_DRAGOON_CKPT_BYTES" ]]; then
echo "ERROR: halogen checkpoint is not $LLM_DRAGOON_CKPT_BYTES bytes (partial download?)" >&2
echo "frank-dragoon: re-check: bash scripts/download-frank-dragoon-flashnext.sh --verify-only" >&2
return 1
fi
for gate in "$LLM_DRAGOON_NGRAM:$LLM_DRAGOON_NGRAM_BYTES:n-gram table" \
"$LLM_DRAGOON_VISION:$LLM_DRAGOON_VISION_BYTES:vision tower"; do
IFS=: read -r gpath gbytes gname <<<"$gate"
if [[ ! -f "$gpath" ]]; then
echo "ERROR: halogen $gname absent at $gpath" >&2
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
return 1
fi
if [[ "$(stat -c %s "$gpath")" != "$gbytes" ]]; then
echo "ERROR: halogen $gname is not $gbytes bytes (partial download?)" >&2
echo "frank-dragoon: re-check: bash scripts/download-frank-dragoon-flashnext.sh --verify-only" >&2
return 1
fi
done
if [[ -L "$LLM_DRAGOON_TOK_DIR" || ! -d "$LLM_DRAGOON_TOK_DIR" ]]; then
echo "ERROR: tokenizer dir missing or a symlink at $LLM_DRAGOON_TOK_DIR" >&2
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
return 1
fi
for tf in $LLM_DRAGOON_TOK_FILES; do
if [[ ! -f "$LLM_DRAGOON_TOK_DIR/$tf" || -L "$LLM_DRAGOON_TOK_DIR/$tf" ]]; then
echo "ERROR: tokenizer file missing or a symlink: $LLM_DRAGOON_TOK_DIR/$tf" >&2
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
return 1
fi
done
# Refuse rather than restart-loop; every gate here is the SAME library
# call the serve script makes, so the two twins cannot drift (the
# 2026-10-01 crash-loop lesson).
# Capture podman output before grepping is moot: require_image is one
# command, no pipe (the 2026-09-30 SIGPIPE lesson).
export GATE_PREFIX=frank-dragoon
require_image "$LLM_DRAGOON_IMAGE" \
"pull it: podman pull docker://${LLM_DRAGOON_IMAGE%@*}" || return 1
require_bytes "$LLM_DRAGOON_CKPT" checkpoint "$LLM_DRAGOON_CKPT_BYTES" \
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh; re-check: --verify-only" || return 1
require_bytes "$LLM_DRAGOON_NGRAM" n-gram "$LLM_DRAGOON_NGRAM_BYTES" \
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh; re-check: --verify-only" || return 1
require_bytes "$LLM_DRAGOON_VISION" vision "$LLM_DRAGOON_VISION_BYTES" \
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh; re-check: --verify-only" || return 1
# shellcheck disable=SC2086
require_dir_files "$LLM_DRAGOON_TOK_DIR" tokenizer \
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" \
$LLM_DRAGOON_TOK_FILES || return 1
retire_halogen_dragoon_quadlet
# The predecessor on a re-deploy is THIS unit: stop it (ExecStop stops
# the container) before waiting on the port. The stray sweep is keyed
# on the FULL container name, which appears in the argv of a manual
# `podman run --name frank-dragoon-halogen` / `podman wait` but NOT in
# this deployer's own command line (`bash deploy-remote.sh
# frank-dragoon`) — so it cannot shoot itself in the foot (the fleet
# pkill law), and it can never match frank-zealot's llama-server.
# on the FULL container name: it appears in the argv of a manual
# `podman run --name frank-dragoon-halogen` / `podman wait`, while a
# pattern naming only the runtime would ALSO match this deployer's own
# command line — a pkill whose pattern hits its own argv takes the
# running deploy down with the target. The full name can never match
# frank-zealot's llama-server either.
run_privileged systemctl stop frank-dragoon.service 2>/dev/null || true
pkill -f 'frank-dragoon-halogen' 2>/dev/null || true
podman rm -f frank-dragoon frank-dragoon-halogen >/dev/null 2>&1 || true
# Nothing may start until the predecessor has released the port
# (2026-09-16: "Deactivated successfully" was not "port free").
wait_port_free 7770 || return 1
relink_unit frank-dragoon
run_privileged systemctl daemon-reload
run_privileged systemctl enable frank-dragoon.service
run_privileged systemctl restart frank-dragoon.service
wait_for_health frank-dragoon http://127.0.0.1:7770/health 1200
start_root_unit frank-dragoon 7770 1200
}
# deploy_llm_zealot — frank-zealot is llama.cpp (master build, CUDA sm_86) on
@@ -1243,37 +1220,18 @@ LLM_ZEALOT_MMPROJ="${LLM_ZEALOT_MMPROJ:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K
deploy_llm_zealot() {
# Refuse rather than restart-loop: a unit whose runtime or payload is
# missing never becomes healthy, and a green deploy would leave the
# endpoint dark with nothing in the log to say why.
if [[ ! -x "$LLM_ZEALOT_RUNTIME" ]]; then
echo "ERROR: llama.cpp runtime absent at $LLM_ZEALOT_RUNTIME" >&2
echo "frank-zealot: build it (cmake -B build-cuda -DGGML_CUDA=ON" >&2
echo "frank-zealot: -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13" >&2
echo "frank-zealot: && cmake --build build-cuda --target llama-server) first." >&2
return 1
fi
for f in "$LLM_ZEALOT_MODEL_FILE" "$LLM_ZEALOT_MMPROJ"; do
if [[ ! -f "$f" ]]; then
echo "ERROR: frank-zealot payload file absent at $f" >&2
echo "frank-zealot: run scripts/download-frank-zealot.sh first;" >&2
echo "frank-zealot: refusing to start a unit that cannot load." >&2
return 1
fi
done
relink_unit frank-zealot
run_privileged systemctl daemon-reload
run_privileged systemctl enable frank-zealot.service
run_privileged systemctl restart frank-zealot.service
wait_for_health frank-zealot http://127.0.0.1:7771/health 900 || return 1
# False-green hardening (frank-dragoon's lesson): the port answering
# proves nothing about WHO answers — the unit must be active with a
# live MainPID, or a stray SGLang holdover is masquerading as deployed.
local main_pid
main_pid="$(run_privileged systemctl show -p MainPID --value frank-zealot.service)"
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
echo "FAIL: :7771 answered but frank-zealot.service has no MainPID — a stray process holds the port" >&2
return 1
fi
echo "frank-zealot: health answered by unit MainPID $main_pid"
# endpoint dark with nothing in the log to say why. Same library calls
# as serve-frank-zealot.sh, so the deployer's twin refuses identically.
export GATE_PREFIX=frank-zealot
require_exec "$LLM_ZEALOT_RUNTIME" runtime \
"build it: cmake -B build-cuda -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86" \
|| return 1
require_file "$LLM_ZEALOT_MODEL_FILE" gguf \
"fetch it: bash scripts/download-frank-zealot.sh" || return 1
require_file "$LLM_ZEALOT_MMPROJ" mmproj \
"fetch it: bash scripts/download-frank-zealot.sh" || return 1
start_root_unit frank-zealot 7771 900
}
# deploy_llm_archon — frank-archon is the radiance engine (StillDeadcode,
@@ -1300,11 +1258,16 @@ LLM_ARCHON_LEGACY_CONTAINERS="frank-archon-qwen frank-archon-r9v qwen-fp8 glm-pr
deploy_llm_archon() {
# Refuse rather than restart-loop: a unit whose image or payload is
# missing never becomes healthy, and a green deploy would leave the
# endpoint dark with nothing in the log to say why.
local env_file="/home/dank/serverAppConfigs/scripts/frank-archon.env"
local image model_dir model_file rad_min mib
# endpoint dark with nothing in the log to say why. The gates are the
# shared library's, called with the values the unit's own env file
# declares — so serve-frank-archon.sh and this handler cannot disagree,
# and the payload path lives in exactly one file.
export GATE_PREFIX=frank-archon
local env_file="${LLM_ARCHON_ENV_FILE:-$REMOTE_DIR/scripts/frank-archon.env}"
local image model_dir model_file rad_min
if [[ ! -f "$env_file" ]]; then
echo "ERROR: frank-archon env file absent: $env_file" >&2
echo "frank-archon: env: missing at $env_file" >&2
echo " hint: deploy the repo checkout first ($REMOTE_DIR)" >&2
return 1
fi
image="$(sed -n 's/^FRANK_ARCHON_IMAGE=//p' "$env_file" | tail -1)"
@@ -1312,28 +1275,16 @@ deploy_llm_archon() {
model_file="$(sed -n 's/^FRANK_ARCHON_MODEL_FILE=//p' "$env_file" | tail -1)"
rad_min="$(sed -n 's/^FRANK_ARCHON_RAD_MIN_MIB=//p' "$env_file" | tail -1)"
if [[ -z "$image" || -z "$model_dir" || -z "$model_file" || -z "$rad_min" ]]; then
echo "ERROR: frank-archon.env missing FRANK_ARCHON_IMAGE/MODEL_DIR/MODEL_FILE/RAD_MIN_MIB" >&2
return 1
fi
# The image gate is the digest ref itself: a name@sha256 ref exists in
# storage only if THAT manifest was pulled. One command — no
# pipefail/grep race (the 2026-09-30 lesson).
if ! podman image exists "$image"; then
echo "ERROR: runtime image absent from storage: $image" >&2
echo "frank-archon: podman pull docker://stilldeadcode/radiance:latest" >&2
return 1
fi
if [[ ! -f "$model_dir/$model_file" ]]; then
echo "ERROR: .rad container missing: $model_dir/$model_file" >&2
echo "frank-archon: run scripts/build-frank-archon-rad.sh" >&2
return 1
fi
mib=$(( $(stat -c %s "$model_dir/$model_file") / 1048576 ))
if [[ "$mib" -lt "$rad_min" ]]; then
echo "ERROR: .rad container too small: $model_dir/$model_file is ${mib} MiB, want >= $rad_min" >&2
echo "frank-archon: rebuild it (scripts/build-frank-archon-rad.sh --force)" >&2
echo "frank-archon: env: missing FRANK_ARCHON_IMAGE/MODEL_DIR/MODEL_FILE/RAD_MIN_MIB" >&2
echo " hint: scripts/frank-archon.env" >&2
return 1
fi
require_image "$image" \
"pull it: podman pull docker://stilldeadcode/radiance:latest (as dank)" \
|| return 1
require_min_mib "$model_dir/$model_file" ".rad" "$rad_min" \
"convert it: scripts/convert-swift15-w4dn8.sh (rad-info is the record)" \
|| return 1
# A stale container with the same name makes `podman run --name` fail at
# start; remove any leftover from a failed previous attempt.
@@ -1352,41 +1303,15 @@ deploy_llm_archon() {
# (2026-09-26) and the frank-dragoon cutover (2026-09-16) both failed
# exactly here when the next engine bound 7770 a second too early.
wait_port_free 7770 || return 1
relink_unit frank-archon
run_privileged systemctl daemon-reload
run_privileged systemctl enable frank-archon.service
run_privileged systemctl restart frank-archon.service
# Cold start: .rad load + PLE pin + expert host-pool upload (the
# publisher's compose healthcheck allows a 15-minute start_period).
# Parity with every predecessor on this port. The HTTP listener binds
# AFTER the full load (main.cpp: engine.start() then engine.serve()),
# so during load the port REFUSES connections — a bare curl is honest
# here, unlike FreeToken's 200-while-loading body. (The MainPID check
# below still guards the stray-port case.)
local i ok=0
for ((i=1; i<=900; i++)); do
if curl -fsS http://127.0.0.1:7770/health 2>/dev/null >/dev/null; then
echo "frank-archon engine healthy after approximately $((i * 2))s"
ok=1
break
fi
sleep 2
done
if [[ "$ok" != 1 ]]; then
echo "FAIL: frank-archon /health never answered 200 in 1800s" >&2
return 1
fi
# False-green hardening (the 2026-09-26 lesson): a bare curl against the
# port cannot tell WHO answers — a stray probe holding :7770 satisfies
# it while the unit is stopped. The deploy is only green if the unit
# itself is active with a live MainPID behind the health answer.
local main_pid
main_pid="$(run_privileged systemctl show -p MainPID --value frank-archon.service)"
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
echo "FAIL: :7770 answered but frank-archon.service has no MainPID — a stray process holds the port" >&2
return 1
fi
echo "frank-archon: health answered by unit MainPID $main_pid"
# publisher's compose healthcheck allows a 15-minute start_period) —
# the 900-iteration budget is 1800s, parity with every predecessor on
# this port. The HTTP listener binds AFTER the full load (main.cpp:
# engine.start() then engine.serve()), so during load the port REFUSES
# connections — a bare curl is honest here, unlike FreeToken's
# 200-while-loading body. (start_root_unit's MainPID check still
# guards the stray-port case.)
start_root_unit frank-archon 7770 900
}
wait_for_health() {
@@ -1586,7 +1511,12 @@ deploy_service() {
tautulli) deploy_tautulli ;;
prometheus|grafana|loki|alloy|node-exporter|alertmanager) deploy_observability "$service" ;;
comfyui) deploy_comfyui ;;
*) deploy_compose_service "$service" ;;
*)
# Every deployable service on every target has a handler above
# (inventory = allow-list, so nothing unreachable arrives).
echo "no handler for service: $service" >&2
exit 2
;;
esac
}
+7 -25
View File
@@ -131,31 +131,13 @@ else
fi
valid_service() {
local service="$1"
case "$TARGET:$service" in
dankclaw:caddy|dankclaw:gluetun|dankclaw:searxng-network|dankclaw:searxng-valkey|\
dankclaw:searxng|dankclaw:marreta|dankclaw:pihole|dankclaw:open-webui|dankclaw:gitea-postgres|\
dankclaw:gitea-server|dankclaw:homepage|\
dankclaw:comfyui|dankclaw:hermes|dankclaw:hermes-dashboard|\
dankclaw:hermes-webui|dankclaw:prometheus|dankclaw:grafana|dankclaw:loki|dankclaw:alloy|\
dankclaw:alertmanager|\
dankclaw:node-exporter|dankclaw:authentik-postgres|dankclaw:authentik-server|\
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|dankclaw:seerr|\
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
dankserver:qbittorrent|dankserver:radarr|dankserver:sonarr|\
dankserver:lidarr|dankserver:readarr|dankserver:node-exporter|\
dankserver:frank-archon|\
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
frankbook-pro:readarr)
return 0
;;
*)
return 1
;;
esac
# The inventory is the only allow-list (see scripts/service-map.py);
# service-map's "allows" exit code is the answer. The former literal
# case list here drifted from reality (it still listed the *arr stack
# under dankserver, which moved to frankbook-pro, and a
# "searxng-network" name that is a generated systemd unit, not a
# deployable service).
python3 "$REPO_ROOT/scripts/service-map.py" allows --host "$TARGET" --service "$1" >/dev/null
}
for service in "${EXPLICIT_SERVICES[@]}"; do
+1 -1
View File
@@ -2,7 +2,7 @@
"""derive-protected-experts.py — rewrite the recipe's protected-expert section
from a calibration recording. RUNS ON DANKSERVER (venv: numpy).
The law (docs/MOE-W4.md "Protected experts", restated in the recipe):
The selection rule (docs/MOE-W4.md "Protected experts", restated in the recipe):
an expert whose share of its layer's down-input energy is at least 3% AND
at least three times its share of the routing is kept bf16; the count must
be EVEN on each rank's half of the layer (the four-bit GEMM takes the rest
+5 -4
View File
@@ -14,10 +14,11 @@
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
# listing/resume downloader, curl+jq only): per-file size checks against
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
# Verify (default) is the AGENTS.md law in three parts: every listed file
# present with the listed size, the shard count matches the weight-map
# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs
# only the public API.
# Verify (default) is three independent agreements: every listed file
# present with the listed size (catches truncation), the shard count
# matching the weight-map index (catches omissions), and every shard's
# sha256 equal to the Hub's LFS oid (catches substituted content).
# Verify needs only the public API.
set -euo pipefail
REPO="ukisai/Swift1.5-Qwen3.8-Flash-Next"
+5 -4
View File
@@ -12,10 +12,11 @@
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
# listing/resume downloader, curl+jq only): per-file size checks against
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
# Verify (default) is the AGENTS.md law in three parts: every listed file
# present with the listed size, the shard count matches the weight-map
# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs
# only the public API.
# Verify (default) is three independent agreements: every listed file
# present with the listed size (catches truncation), the shard count
# matching the weight-map index (catches omissions), and every shard's
# sha256 equal to the Hub's LFS oid (catches substituted content).
# Verify needs only the public API.
set -euo pipefail
REPO="Qwen/Qwen3.8-Flash-Next-FP8"
+11 -6
View File
@@ -4,12 +4,15 @@
# .hgn, the REQUIRED lookup table and the optional vision tower from
# peonist-ai/halogen-qwen3.8-flash-next (both at the revisions the model
# card pins), and the flat tokenizer the two repos share. All public, no
# token — and per the fleet secret law, none would ever touch argv.
# token needed — and a secret would never be passed via argv anyway: the
# command line is world-readable through /proc, so any token placed there
# leaks to every process-listing tool on the host.
#
# The fetch law, same as every sibling script: curl -f (an access error
# must not become a "successful" 153-byte artifact), exact byte count from
# the Hub tree API, LFS sha256 verification, --verify-only to re-check an
# existing payload without downloading.
# The fetch rules, uniform across every sibling script: curl -f so an
# access error fails loudly instead of writing the error body as a
# "successful" tiny artifact; the exact byte count from the Hub tree API;
# LFS sha256 verification; --verify-only to re-check an existing payload
# without downloading.
#
# File manifest (verified against the Hub tree API on 2026-10-06):
# qwen38-flash-next-v2-swift15-abliterated.hgn 66,687,678,464 B sha256
@@ -26,7 +29,9 @@
# tokenizer/* @e053f488 — byte-identical to the 27B-era tokenizer
# already on disk (tokenizer.json LFS
# 0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3);
# fetched fresh rather than symlinked, per the no-symlink law.
# fetched fresh, never symlinked: a snapshot link points into the
# host-side cache directory, which does not exist inside the
# container mount, so the link dangles exactly where it is consumed.
set -euo pipefail
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}"
+8 -6
View File
@@ -1,13 +1,15 @@
#!/usr/bin/env bash
# download-frank-dragoon-halogen.sh — fetch frank-dragoon's halogen payload:
# the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the
# PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and per the fleet
# secret law, none would ever touch argv).
# PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and a token
# would never travel in argv: command lines are world-readable through
# /proc, so a secret placed there leaks to any process-listing tool).
#
# The fetch law, same as every sibling script: curl -f (an access error
# must not become a "successful" 153-byte artifact), exact byte count from
# the model card, LFS sha256 verification, --verify-only to re-check an
# existing payload without downloading.
# The fetch rules, uniform across every sibling script: curl -f so an
# access error fails loudly instead of writing the error body as a
# "successful" tiny artifact; the exact byte count from the model card;
# LFS sha256 verification; --verify-only to re-check an existing payload
# without downloading.
set -euo pipefail
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-qwen3.8-27b}"
+7 -7
View File
@@ -11,13 +11,13 @@ Tensor extents are DERIVED from consecutive GGUF header offsets — no quant
type-size table (those are easy to get subtly wrong; a wrong per-element size
silently under-covers each tensor and dumps tail samples into a junk bucket).
Fleet law this instrument established (2026-09-26, 75 s window, 4 mixed
streams, 2.7M expert-tensor samples): per-BLOCK read volume is FLAT
(0.93x-1.08x of mean across all 40 CPU-resident blocks) because MoE top-k
routing fires every block every token — there are no "most active layers" at
`-ot` granularity. The real skew is per-EXPERT (top 10% of (block,expert)
cells carry ~40% of samples), which tensor-level overrides cannot reach.
Any 8 GPU-resident blocks are therefore equivalent; COUNT is the only lever.
What this instrument settles: per-BLOCK expert read volume is FLAT across
all CPU-resident blocks, because MoE top-k routing fires every block on
every token — there are no "most active layers" at `-ot` granularity. The
real skew is per-EXPERT: a small fraction of (block,expert) cells carries
most of the samples, and tensor-level overrides cannot reach that level.
Any fixed number of GPU-resident blocks is therefore equivalent; COUNT is
the only lever.
Usage (needs passwordless sudo for perf only; model files must be readable):
fp-expert-activity.py <pid> [duration_s] # dump + sample + analyze
+14 -12
View File
@@ -115,15 +115,16 @@ FRANK_ARCHON_TP=2
FRANK_ARCHON_TP_WIRE=wht6
# Decode slots: TWO, so the served window is a CAPACITY CONTRACT — two
# concurrent conversations of --max-model-len fit the KV pool (see the KV
# block below). The "seqs 2 cannot boot" law is NOT radiance's: it was the
# tcclaviger vLLM fork's torch.compile shape specialization on
# query_start_loc (ConstraintViolationError, measured 2026-10-02, retired
# with that runtime). Radiance booted seqs 2 at the fp8-native first boot,
# and the 2026-10-06 decode sweep measured seqs 2 and 3 statistically tied
# (30.9-33.1 t/s, inside the +-3% noise floor). Side benefit: the fused TP
# all-reduce is declared over seqs x (1 + spec) = 6 rows, under the 25-row
# bound at which the wht6 wire stays EXACT (single-sequence decode and any
# step of <= 25 rows is bit-exact; only busier steps are lossy).
# block below). Slot-count limits observed on other runtimes do NOT
# transfer: a "this slot count cannot boot" refusal is usually that
# runtime's compiler specializing shapes (e.g. a ConstraintViolationError
# from shape specialization on query_start_loc), not a property of the
# engine now running. This engine boots seqs 2, and seqs 2
# vs 3 decode measurements land inside each other's noise floor. Side
# benefit: the fused TP all-reduce is declared over seqs x (1 + spec) = 6
# rows, under the 25-row bound at which the wht6 wire stays EXACT
# (single-sequence decode and any step of <= 25 rows is bit-exact; only
# busier steps are lossy).
FRANK_ARCHON_MAX_SEQS=2
# Served window. Native is 262,144; 200,000 is the publisher's production
# value (compose) and what models.yml's contextWindow must quote — a longer
@@ -203,9 +204,10 @@ FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
# (104 GiB available with the service loaded).
FRANK_ARCHON_PREFIX_HOST_MIB=8192
FRANK_ARCHON_PREFIX_DISK_MIB=65536
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
# RAM the cards read. `ram` REFUSES the start if the memlock limit is too
# small — the unit's LimitMEMLOCK=infinity is the other half of this law.
# The PLE n-gram table (47.7 GiB) serves from pinned host RAM the cards
# read. `ram` REFUSES the start if the memlock limit is too small — a
# named refusal beats a mid-load allocator failure — and the unit's
# LimitMEMLOCK=infinity is the other half of the arrangement.
FRANK_ARCHON_NGRAM_PLACEMENT=ram
# TTM preflight: the pinned ledger (2 x 44 GiB pools + 47.7 GiB PLE + 8 GiB
# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
+9 -7
View File
@@ -42,10 +42,11 @@
# form of the engine's default, so the seam test can pin the product.
# KV costs ~28 KiB/position (FLAGS.md) => 524,288 x 28 KiB = 14.0 GiB.
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
# drafter is NOT slot-locked on this engine (batching and speculation
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own
# quickstart ships 2 slots WITH the head), so the second slot is
# concurrent capacity, not a speculation tax.
# drafter is NOT slot-locked on this engine. A "slots > 1 disables the
# drafter" restriction is a property of the engine generation that
# imposed it, not of speculation itself — batching and speculation
# coexist here, and the card's own quickstart ships 2 slots WITH the
# head — so the second slot is concurrent capacity, not a speculation tax.
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
# mmap'd page cache beside the checkpoint, not a pinned charge — the
# card's own quickstart arithmetic does not bill it):
@@ -88,9 +89,10 @@
# before. Kernel needs CONFIG_HSA_AMD_SVM (CachyOS ships it) and
# amdgpu.noretry=0 (check at deploy).
# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest
# is what was qualified). 0.16.2 = the version the model card names as its
# quickstart pin; RepoDigest captured at pull on 2026-10-06.
# Image: digest-pinned, not tag-pinned — a tag can be re-pointed to a
# build that was never qualified here, while the digest pins exactly the
# build that was tested. 0.16.2 = the version the model card names as its
# quickstart pin; the RepoDigest was captured at a pull of that tag.
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
# Container identity: kept from the 27B era (unit ExecStop names it).
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
+5 -5
View File
@@ -16,11 +16,11 @@
# custom-quant lock-in like the ROCmFPX era). NO nextn/mtp tensors exist, so
# in-file MTP is impossible here; SPEC_TYPE=none is forced by the payload.
#
# KV rate law: 8 full layers × 4 KV heads × 256 dim = 8,192 elements per
# token per cache. f16 K + f16 V = 32,768 B/token = 32 KiB/token, so the
# 401,408-token pool is 12.25 GiB. The 24 DeltaNet layers cost a fixed
# recurrent state per SLOT (not per token) — that is why two 200k slots fit
# where the 26B MoE fit only one.
# The KV rate follows from the architecture: 8 full layers × 4 KV heads ×
# 256 dim = 8,192 elements per token per cache. f16 K + f16 V = 32,768
# B/token = 32 KiB/token, so the 401,408-token pool is 12.25 GiB. The 24
# DeltaNet layers cost a fixed recurrent state per SLOT (not per token) —
# that is why two 200k slots fit where the 26B MoE fit only one.
#
# Feasibility comes from the predecessor's MEASURED boots, not a formula:
# MiMo-V2.6-Distill-Qwen-9B (identical qwen35 geometry, 7,793,710,624 B) at
+6 -1
View File
@@ -12,7 +12,12 @@ Group=dank
LimitMEMLOCK=infinity
Environment="FRANK_ZEALOT_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-zealot.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-zealot.sh
Restart=on-failure
# Restart=always is the right policy for a serving daemon: a process that
# exits on its own — exit 0 — is a CLEAN stop to systemd, so on-failure
# never fires and the endpoint stays dark (with the unit reporting
# success) until the next boot. frank-zealot is the fleet's compaction
# hub — every endpoint's compaction dies when it does.
Restart=always
RestartSec=10
[Install]
+133
View File
@@ -0,0 +1,133 @@
#!/usr/bin/env bash
# llm-gates.sh — the artifact gates every LLM endpoint runs before it starts.
#
# Why this file exists: every LLM endpoint gates its payload twice — the
# serve script at start, the deployer before dispatch. Two hand-written
# copies of the same checks drift apart in wording and strictness, and the
# drift surfaces as deploy-time disagreement: one side passes, the other
# refuses, and the failure lands in production instead of in the gate.
# Both sides source this library, so the refusal is one text and one rule.
#
# Contract (public interface):
# require_exec PATH LABEL HINT
# require_file PATH LABEL HINT
# require_bytes PATH LABEL WANT_BYTES HINT
# require_min_mib PATH LABEL MIN_MIB HINT
# require_dir_files DIR LABEL HINT FILE...
# require_image REF HINT
#
# Each returns 0 and prints nothing on success. On failure it prints two
# lines to stderr — the first naming the endpoint ($GATE_PREFIX) and the
# artifact, the second the remedy — and returns 1. It NEVER exits: the serve
# scripts run under `set -e` (so a refusal is terminal), while the deployer
# wants `|| return 1` so a failed gate reports a red deploy for that one
# service and the dispatcher's state marker stays pending.
# Lines are labelled by ARTIFACT CLASS, and the class is the whole label
# (the path rides in the same line): the class + path is what an operator
# acts on, so no wording order and no payload rename can make the two twins
# disagree. The deployer's old "halogen vision tower" and the serve script's
# "vision-tower" were one gate in two vocabularies that no grep could join.
# Size checks use equality, not a floor, for `require_bytes`: a fetch that
# truncates looks like a success (an HTTP error exit is often swallowed, and
# the error body is written in the artifact's place), so "exists and is big
# enough" is not proof of the right artifact. The declared byte count is a
# contract in both directions — a bigger file is a DIFFERENT artifact.
# GATE_PREFIX names the endpoint in every refusal (e.g. frank-dragoon). The
# caller exports it; shellcheck cannot follow it across the source boundary,
# so the read site carries the directive.
# shellcheck disable=SC2034
: "${GATE_PREFIX:=gate}"
# Two bash behaviors make the `return 1` at every call site load-bearing:
# * a function whose body ends inside an `if` returns the CONDITION's
# status (true = 0), so a gate that ends on its own refusal prints the
# refusal and reports SUCCESS;
# * `set -e` is suspended for any command in a `… || …` chain, so a gate
# that runs inside one must not be trusted to abort the caller.
# Every gate's refusal therefore carries an explicit `|| return 1`, and the
# caller consumes the status (`|| exit 1`, `|| return 1`).
_gate_fail() {
printf '%s: %s\n' "$GATE_PREFIX" "$1" >&2
printf ' hint: %s\n' "$2" >&2
return 1
}
# require_file — PATH must be a regular file. A directory (an unpacked
# staging dir wearing the payload's name) and a symlink (a HuggingFace cache
# snapshot's entry, which dangles inside the container mount) both refuse.
require_file() {
local path="$1" label="$2" hint="$3"
if [[ -L "$path" || ! -f "$path" ]]; then
_gate_fail "$label: missing at $path" "$hint" || return 1
fi
return 0
}
# require_exec — PATH must be an executable regular file. The native runtimes
# (llama.cpp's llama-server) are binaries the operator builds; a tree that
# compiled but lost its +x, or a directory wearing the binary's name, must
# refuse the same way a missing one does.
require_exec() {
local path="$1" label="$2" hint="$3"
if [[ -L "$path" || ! -x "$path" ]]; then
_gate_fail "$label: missing at $path" "$hint" || return 1
fi
return 0
}
# require_bytes — PATH must exist AND be exactly WANT_BYTES bytes.
require_bytes() {
local path="$1" label="$2" want="$3" hint="$4" got
require_file "$path" "$label" "$hint" || return 1
got="$(stat -c %s "$path")"
if [[ "$got" != "$want" ]]; then
_gate_fail "$label: is $got bytes, expected $want" "$hint" || return 1
fi
return 0
}
# require_min_mib — PATH must be at least MIN_MIB mebibytes. The floor form
# is for build products (a .rad container is a range of legal sizes; a
# truncated convert is not).
require_min_mib() {
local path="$1" label="$2" min="$3" hint="$4" mib
require_file "$path" "$label" "$hint" || return 1
mib=$(( $(stat -c %s "$path") / 1048576 ))
if (( mib < min )); then
_gate_fail "$label: is $mib MiB, expected >= $min MiB" "$hint" || return 1
fi
return 0
}
# require_dir_files — DIR must be a directory and every named member a
# regular file, never a symlink: snapshot layouts link files into place, and
# a link dangles inside the container mount even when the directory itself
# is real. A symlinked DIRECTORY is refused too — the same layout can link
# the directory itself, and the link dangles identically.
require_dir_files() {
local dir="$1" label="$2" hint="$3"
shift 3
local f
if [[ -L "$dir" || ! -d "$dir" ]]; then
_gate_fail "$label: missing at $dir" "$hint" || return 1
fi
for f in "$@"; do
if [[ -L "$dir/$f" || ! -f "$dir/$f" ]]; then
_gate_fail "$label: $dir/$f is missing or a symlink" "$hint" || return 1
fi
done
return 0
}
# require_image — the image ref must be in podman storage. A name@sha256 ref
# exists only if THAT manifest was pulled, so the gate is the ref itself, in
# one command: no pipefail/grep race (the 2026-09-30 lesson).
require_image() {
local ref="$1" hint="$2"
if ! podman image exists "$ref"; then
_gate_fail "image: $ref is not in podman storage" "$hint" || return 1
fi
return 0
}
@@ -29,10 +29,11 @@
# its base SupportsEagleBase (has_own_lm_head, has_own_embed_tokens) -- and
# getattr_static()s each one off the instance, breaking on the first miss.
# So BOTH wrappers need all three attributes.
# * MRO law: INHERITING SupportsEagle3 as a base dies at class creation with
# "Cannot create a consistent method resolution order (MRO) for bases ..."
# (GLM's wrappers already carry four/five Protocol-derived bases). The
# attribute route avoids the MRO entirely. Also proven by simulation:
# * Why not inheritance: adding SupportsEagle3 as a BASE class dies at
# class creation with "Cannot create a consistent method resolution
# order (MRO) for bases ..." (GLM's wrappers already carry four/five
# Protocol-derived bases; the base ordering cannot take another). The
# attribute route avoids the MRO entirely; simulation confirms that
# the four-base ForCausalLM ordering cannot take SupportsEagle3 in ANY
# position.
# * Method placement: without inheriting, the Protocol's default
+6 -4
View File
@@ -9,7 +9,9 @@ No scratch container, no mask: the service under test is the service.
Three probes, in increasing cost:
A decode at depth — one ~51k-token chat completion, 512 tokens, greedy.
Reports the engine's own `timings` (prefill, decode, draft acceptance).
The bench law applies: >=128 generated tokens, greedy, real text.
Method: >=128 generated tokens (below that, first-token latency
dominates the decode rate), greedy (sampling noise is unattributable),
real text (synthetic text inflates draft acceptance).
B the capacity contract — TWO concurrent chats of ~197k prompt tokens
(~394k tokens resident) each carrying a unique marker at its head. Both
must answer their own marker. This is what a 2 x 200,000 endpoint owes:
@@ -17,11 +19,11 @@ Three probes, in increasing cost:
C prefix-cache restore — probe A's cold prompt again, so the disk tier has
to give the blocks back (cache_n > 0, TTFT collapses).
PROBE LAW (learned the hard way, twice): radiance routes the thinking block to
PROBE PITFALL: radiance routes the thinking block to
`reasoning_content`, so a chat request whose budget is spent inside
`<think>` returns EMPTY `content` with `finish_reason: stop`. A probe that
reads `content` alone scores 0/2 on a healthy server. Every request here sets
chat_template_kwargs.enable_thinking=false and reads both fields.
reads `content` alone scores 0/2 on a healthy server. Every request here
sets chat_template_kwargs.enable_thinking=false and reads both fields.
A/B method for a shape change (no scratch container): write a variant env file
on the host, point a transient systemd drop-in at it
@@ -1,10 +1,12 @@
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official
# base) as a radiance container for frank-archon: EVERY routed expert -- gate_up AND down --
# through GPTQ into the w4nl64a8h table. The plugin's one-table-one-encoding law
# ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served
# beside it", qwen4exp_fp8 rad_arch_declare) forbids the original split tier (down block-fp8 +
# gate_up w4): the served kernel is w4nl64a8h for the whole expert set, bf16 experts beside it.
# through GPTQ into the w4nl64a8h table. The converter declares ONE
# encoding per expert table (qwen4exp_fp8 rad_arch_declare): the served
# kernel covers the whole expert set, and the only legal mixed layout is a
# WHOLE expert kept plain bf16 beside it. That rules out the original
# split tier (down block-fp8 + gate_up w4), which asks one table for two
# encodings.
# The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears, block-fp8 hc mixers
# and MTP fc, the fixed-scale n-gram table, the 2-bit draft head.
#
@@ -25,11 +27,14 @@
# The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE
# CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only.
# The Swift 1.5 finetune's routing distribution is not assumed equal: the derivation step rewrites
# this section from Swift's OWN calibration recording before this recipe is ever run, by the base's
# law -- an expert whose share of its layer's down-input energy is at least 3% and at least three
# times its share of the routing, an even count on each rank's half of the layer (the four-bit GEMM
# takes the rest as two equal tables by parity) by adding that half's next most energetic expert.
# The Swift 1.5 finetune's routing distribution is not assumed equal: the
# derivation step rewrites this section from Swift's OWN calibration
# recording before this recipe is ever run. The selection rule is the base
# checkpoint's: an expert whose share of its layer's down-input energy is
# at least 3% AND at least three times its share of the routing; an even
# count on each rank's half of the layer (the four-bit GEMM takes the rest
# as two equal tables by parity), padded with that half's next most
# energetic expert.
# PROTECTED-BEGIN (rewritten from Swift's OWN grams by derive-protected-experts.py)
blk.1.ffn_*_exps.271.weight cast dtype=bf16
blk.1.ffn_*_exps.355.weight cast dtype=bf16
+29 -32
View File
@@ -26,6 +26,14 @@
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# The artifact gates are the shared library: the serve script and the
# deployer gate the same payload, and two hand-written copies of the same
# checks drift apart until one side passes what the other refuses. One
# library, one refusal text, one rule.
export GATE_PREFIX=frank-archon
# shellcheck disable=SC1091
source "$SCRIPT_DIR/lib/llm-gates.sh"
ENV_FILE="${FRANK_ARCHON_ENV_FILE:-$SCRIPT_DIR/frank-archon.env}"
if [ ! -f "$ENV_FILE" ]; then
@@ -65,38 +73,22 @@ set +a
# The image gate is the digest-pinned ref itself: a name@sha256 ref exists
# in storage only if THAT manifest was pulled — a retagged look-alike or a
# newer latest does not satisfy it. `podman image exists` takes a
# name@digest ref in one command — no pipefail/grep race (the 2026-09-30
# lesson). Runs as dank: the unit's process uses dank's ROOTLESS podman
# storage (root unit, User=dank — the fleet's frank-archon shape).
if ! podman image exists "$FRANK_ARCHON_IMAGE"; then
echo "ERROR: radiance image absent: $FRANK_ARCHON_IMAGE" >&2
echo "frank-archon: podman pull docker://stilldeadcode/radiance:latest (as dank)" >&2
exit 1
fi
# newer latest does not satisfy it. One command, no pipefail/grep race (the
# 2026-09-30 lesson). Runs as dank: the unit's process uses dank's ROOTLESS
# podman storage (root unit, User=dank — the fleet's frank-archon shape).
require_image "$FRANK_ARCHON_IMAGE" \
"podman pull docker://stilldeadcode/radiance:latest (as dank)" || exit 1
# The payload gate: the .rad container file, sized. A truncated convert
# would otherwise buy a guaranteed long-timeout boot instead of this fast
# failure. The staging dir / raw checkpoint are NOT this script's concern
# (scripts/build-frank-archon-rad.sh built the file; its rad-info output
# is the record).
RAD="$FRANK_ARCHON_MODEL_DIR/$FRANK_ARCHON_MODEL_FILE"
if [ ! -f "$RAD" ]; then
echo "ERROR: .rad container missing: $RAD" >&2
echo "frank-archon: run scripts/build-frank-archon-rad.sh" >&2
exit 1
fi
RAD_MIB=$(( $(stat -c %s "$RAD") / 1048576 ))
if [ "$RAD_MIB" -lt "$FRANK_ARCHON_RAD_MIN_MIB" ]; then
echo "ERROR: .rad container too small: $RAD is ${RAD_MIB} MiB, want >= $FRANK_ARCHON_RAD_MIN_MIB" >&2
echo "frank-archon: the convert was truncated — rebuild it" >&2
exit 1
fi
# (the convert script built the file; its rad-info output is the record).
require_min_mib "$FRANK_ARCHON_MODEL_DIR/$FRANK_ARCHON_MODEL_FILE" \
container "$FRANK_ARCHON_RAD_MIN_MIB" \
"build it: scripts/convert-swift15-w4dn8.sh (rad-info is the record)" || exit 1
if [ ! -f "$FRANK_ARCHON_TEMPLATE" ]; then
echo "ERROR: chat template missing: $FRANK_ARCHON_TEMPLATE" >&2
exit 1
fi
require_file "$FRANK_ARCHON_TEMPLATE" template \
"the chat template is vendored: scripts/qwen38-froggeric-v22.5.jinja" || exit 1
mkdir -p "$FRANK_ARCHON_STATE_DIR/kvcache/frank-archon"
# The engine's create is NOT recursive: with only the state dir mounted, its
@@ -112,8 +104,12 @@ mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
# that wrote it. The dir is pre-created for the same reason as kvcache
# (the engine's create is not recursive).
if [ ! -e /dev/kfd ] || [ ! -d /dev/dri ]; then
echo "ERROR: GPU device nodes absent (/dev/kfd, /dev/dri)" >&2
# GPU access is mandatory; the node paths are knobs so the seam can drive
# this gate on a box without AMD GPUs (production defaults).
DEV_KFD="${FRANK_ARCHON_DEV_KFD:-/dev/kfd}"
DEV_DRI="${FRANK_ARCHON_DEV_DRI:-/dev/dri}"
if [ ! -e "$DEV_KFD" ] || [ ! -d "$DEV_DRI" ]; then
echo "ERROR: GPU device nodes absent ($DEV_KFD, $DEV_DRI)" >&2
exit 1
fi
@@ -129,8 +125,9 @@ ttm_cap_mib() {
# min over the knobs THIS kernel exposes — the current kernel has
# pages_limit only (no pages_max); a knob that is absent is skipped,
# not a zero (a zero would refuse a healthy cap).
local v cur=""
for p in /sys/module/ttm/parameters/pages_limit /sys/module/ttm/parameters/pages_max; do
local v cur="" dir
dir="${FRANK_ARCHON_TTM_SYSFS:-/sys/module/ttm/parameters}"
for p in "$dir/pages_limit" "$dir/pages_max"; do
[ -r "$p" ] || continue
v="$(cat "$p" 2>/dev/null)" || continue
[ -n "$v" ] || continue
@@ -155,7 +152,7 @@ for CTR in $FRANK_ARCHON_LEGACY_CONTAINERS "$FRANK_ARCHON_CONTAINER"; do
podman rm -f "$CTR" >/dev/null 2>&1 || true
done
LOG=/home/dank/logs/frank_archon.log
LOG="${FRANK_ARCHON_LOG:-/home/dank/logs/frank_archon.log}"
mkdir -p "$(dirname "$LOG")"
# Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
+34 -43
View File
@@ -29,7 +29,14 @@
# exactly like frank-archon's.
set -euo pipefail
# The artifact gates are the shared library — the serve script and the
# deployer refuse the same missing/truncated payload with the same words
# (the 2026-10-01 crash-loop was the two hand-written twins disagreeing).
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
export GATE_PREFIX=frank-dragoon
# shellcheck disable=SC1091
source "$SCRIPT_DIR/lib/llm-gates.sh"
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
if [ ! -f "$ENV_FILE" ]; then
echo "frank-dragoon: env file not found: $ENV_FILE" >&2
@@ -57,55 +64,38 @@ set +a
: "${HALOGEN_KV_SLOTS:?HALOGEN_KV_SLOTS must be set}"
: "${HALOGEN_SPEC_ADAPT:?HALOGEN_SPEC_ADAPT must be set (0 = head always on)}"
# Capture podman's output BEFORE grepping: under `set -o pipefail` the
# `podman images | grep -q` pattern races (grep exits on first match,
# podman dies SIGPIPE 141, pipefail turns the 141 into "no match" — the
# 2026-09-30 deploy refusal).
DIGESTS="$(podman images --format '{{.RepoDigests}}')"
if ! grep -qF "${FRANK_DRAGOON_IMAGE#*@}" <<<"$DIGESTS"; then
echo "frank-dragoon: image $FRANK_DRAGOON_IMAGE not in podman storage;" >&2
echo " pull it: podman pull ${FRANK_DRAGOON_IMAGE%@*}" >&2
exit 1
fi
# The image gate is the digest-pinned ref itself: `podman image exists` takes
# a name@sha256 ref in one command, so a retagged look-alike or a newer
# latest cannot satisfy it and there is no pipefail/SIGPIPE race with grep
# (the 2026-09-30 deploy refusal). Runs as dank: the unit's process uses
# dank's rootless storage.
require_image "$FRANK_DRAGOON_IMAGE" \
"pull it: podman pull docker://$FRANK_DRAGOON_IMAGE" || exit 1
# Size-exact gates on all three payload files: a truncated curl exits 0 on
# some failures and a partial .hgn loads far enough to look alive, so the
# byte counts from the Hub tree API are the contract (fleet fetch law).
ck() {
# ck <env-name> <path> <expected-bytes>
if [ ! -f "$2" ]; then
echo "frank-dragoon: $1 $2 missing;" >&2
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
exit 1
fi
local sz
sz="$(stat -c %s "$2")"
if [ "$sz" != "$3" ]; then
echo "frank-dragoon: $1 is $sz bytes, expected $3 (partial download?)" >&2
exit 1
fi
}
ck checkpoint "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_CHECKPOINT" "$FRANK_DRAGOON_CHECKPOINT_BYTES"
ck n-gram-table "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_NGRAM_TABLE" "$FRANK_DRAGOON_NGRAM_BYTES"
ck vision-tower "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_VISION_TOWER" "$FRANK_DRAGOON_VISION_BYTES"
# Size-exact gates on all three payload files: a truncated download can
# exit 0, and a partial .hgn loads far enough to look alive, so existence
# proves nothing. The byte count published by the Hub tree API is the
# contract: any other size is a different (broken) artifact.
require_bytes "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_CHECKPOINT" \
checkpoint "$FRANK_DRAGOON_CHECKPOINT_BYTES" \
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" || exit 1
require_bytes "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_NGRAM_TABLE" \
n-gram "$FRANK_DRAGOON_NGRAM_BYTES" \
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" || exit 1
require_bytes "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_VISION_TOWER" \
vision "$FRANK_DRAGOON_VISION_BYTES" \
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" || exit 1
# The payload ships the tokenizer as a FLAT DIRECTORY. A HuggingFace cache
# snapshot symlinks each file into a sibling blobs/, and those symlinks
# dangle inside the container mount — so the directory must be real and
# every named file must be a regular file, not a link.
TOK_DIR="$FRANK_DRAGOON_MODEL_DIR/tokenizer"
if [ -L "$TOK_DIR" ] || [ ! -d "$TOK_DIR" ]; then
echo "frank-dragoon: tokenizer dir $TOK_DIR missing or a symlink (HF cache snapshot?);" >&2
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
exit 1
fi
for TF in $FRANK_DRAGOON_TOKENIZER_FILES; do
if [ ! -f "$TOK_DIR/$TF" ] || [ -L "$TOK_DIR/$TF" ]; then
echo "frank-dragoon: tokenizer file $TOK_DIR/$TF missing or a symlink;" >&2
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
exit 1
fi
done
# The tokenizer file list is word-separated ON PURPOSE (one argument per
# file), so the unquoted expansion below is the interface, not a slip.
# shellcheck disable=SC2086
require_dir_files "$FRANK_DRAGOON_MODEL_DIR/tokenizer" tokenizer \
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" \
$FRANK_DRAGOON_TOKENIZER_FILES || exit 1
# GPU access is mandatory but the node paths are variables so the seam can
# drive the gate on a box without an AMD GPU.
@@ -115,6 +105,7 @@ if [ ! -e "$DEV_KFD" ] || [ ! -d "$DEV_DRI" ]; then
echo "frank-dragoon: $DEV_KFD or $DEV_DRI missing — no gfx1151 to serve on" >&2
exit 1
fi
# Host-RAM preflight — the engine's pin gate, checked BEFORE podman run.
# The engine refuses to pin when MemAvailable is below the checkpoint plus
# a 16 GiB floor: pinned pages cannot be swapped, and under the floor the
+21 -8
View File
@@ -28,6 +28,14 @@
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
# The artifact gates are the shared library (scripts/lib/llm-gates.sh): the
# serve script and the deployer gate the same payload, and two hand-written
# copies of the same checks drift apart until one side passes what the
# other refuses.
export GATE_PREFIX=frank-zealot
# shellcheck disable=SC1091
source "$SCRIPT_DIR/lib/llm-gates.sh"
ENV_FILE="${FRANK_ZEALOT_ENV_FILE:-$SCRIPT_DIR/frank-zealot.env}"
if [[ -f "$ENV_FILE" ]]; then
set -a
@@ -62,9 +70,9 @@ fi
# FP8 class; no true fp8 KV dtype exists in this build).
: "${CACHE_TYPE_K:=f16}"
: "${CACHE_TYPE_V:=f16}"
# Speculative decoding is OFF: this Q6_K file carries NO nextn/mtp tensors, so
# there is no in-file head (the law: MTP is a property of the file, not of the
# flags). SPEC_TYPE=none strips BOTH speculative flags (flags move as a set);
# Speculative decoding is OFF: this Q6_K file carries NO nextn/mtp tensors,
# so there is no in-file head — a speculative head is a property of the
# model file, not of the flags; no flag creates tensors that are absent.
# naming a drafter requires SPEC_DRAFT_FILE alongside SPEC_TYPE — the Gemma4
# assistant on disk is arch gemma4-assistant and cannot drafter-match qwen35.
: "${SPEC_TYPE:=none}"
@@ -81,13 +89,18 @@ fi
# Fail loudly rather than half-loading: a missing file is a download that
# has not finished, and Restart=always would turn it into a crash loop that
# reads like a runtime fault.
for path in "$LLAMA_BIN" "$MODEL_FILE"; do
[[ -e "$path" ]] || { echo "frank-zealot: missing $path" >&2; exit 1; }
done
# reads like a runtime fault. The refusal text is the shared gate library's,
# so the deployer's twin refuses identically.
require_file "$LLAMA_BIN" runtime \
"build it: scripts/build-llama-master-cuda.sh" || exit 1
require_file "$MODEL_FILE" gguf \
"fetch it: bash scripts/download-frank-zealot.sh" || exit 1
if [[ -n "$MMPROJ_FILE" ]]; then
require_file "$MMPROJ_FILE" mmproj \
"fetch it: bash scripts/download-frank-zealot.sh" || exit 1
fi
EXTRA_ARGS=()
if [[ -n "$MMPROJ_FILE" ]]; then
[[ -e "$MMPROJ_FILE" ]] || { echo "frank-zealot: missing $MMPROJ_FILE" >&2; exit 1; }
EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
fi
+750
View File
@@ -0,0 +1,750 @@
#!/usr/bin/env python3
"""service-map.py — the only reader of the service inventory.
Four lists used to agree by discipline (the planner's path rules, the
deployer's allow-list, the changed-test mapper, and the scrape/probe/DNS
facts written by hand in three config files). They now all read
`shared/services.yml` through this tool:
hosts the deploy targets
services [--host H] deployables a target accepts, canonical order
allows --host H --service S exit 0 when H accepts S (silent)
plan --host H PATH... services selected by changed paths, order-safe
seams [--host H] PATH... the Bats seams a changed path must re-run
owner --host H PATH the service that owns a file (empty: none)
units --host H --service S "<unit> <unit_dir>" lines the handler installs
stack --host H --service S the deployable + members one handler deploys
get --service S [--host H] F one field as JSON
render pihole-hosts the Pi-hole addn-hosts file
check [NAME...] integrity + drift gates (exit 1 on drift)
Two rules the engine enforces structurally:
* A path selects only services that RUN on the planned target. Path
matching that is host-blind (match first, filter later) lets a file on
one host plan a service on another; selection is scoped to the host the
path belongs to.
* The netns family is DERIVED, never declared: a unit file that says
`Network=gluetun.container` rides that tunnel, so a gluetun change selects
every sharer. The unit files are the source of that coupling.
"""
from __future__ import annotations
import argparse
import json
import subprocess
import sys
from functools import lru_cache
from pathlib import Path, PurePosixPath
from typing import Any
import yaml
REPO_ROOT = Path(__file__).resolve().parent.parent
DEFAULT_INVENTORY = REPO_ROOT / "shared" / "services.yml"
REQUIRED_KEYS = ("name", "hosts", "kind", "tests")
KINDS = {"quadlet", "root-unit", "vm-quadlet", "host"}
UNIT_SUFFIXES = (".container", ".network")
# Repo material that is never a deployed runtime's state.
NON_RUNTIME_ROOTS = {"docs", "report", "old", ".githooks", ".obsidian", ".github",
"tests", "serverAppConfigs-docs"}
NON_RUNTIME_NAMES = {".env.example"}
NON_RUNTIME_PATHS = {"README.md", "AGENTS.md", ".gitignore"}
def is_trash(path: str) -> bool:
return ".AGENTTRASH" in PurePosixPath(path).parts
# --------------------------------------------------------------------------
# inventory
# --------------------------------------------------------------------------
class Inventory:
def __init__(self, doc: dict[str, Any], origin: str) -> None:
self.origin = origin
self.hosts: dict[str, dict[str, Any]] = doc["hosts"]
self.services: list[dict[str, Any]] = doc["services"]
self.unowned: dict[str, str] = doc.get("unowned", {})
for index, svc in enumerate(self.services):
svc["_index"] = index
@classmethod
def load(cls, path: str | Path = DEFAULT_INVENTORY) -> "Inventory":
doc = yaml.safe_load(Path(path).read_text())
if not isinstance(doc, dict) or "services" not in doc:
raise SystemExit(f"{path}: not a service inventory document")
return cls(doc, str(path))
def on(self, host: str) -> list[dict[str, Any]]:
return [s for s in self.services if host in s["hosts"]]
def named(self, host: str, name: str) -> list[dict[str, Any]]:
return [s for s in self.services if s["name"] == name and host in s["hosts"]]
def any_named(self, name: str) -> list[dict[str, Any]]:
return [s for s in self.services if s["name"] == name]
@property
def order(self) -> list[str]:
seen: list[str] = []
for svc in self.services:
if svc["name"] not in seen:
seen.append(svc["name"])
return seen
def edge_ip(self) -> str:
"""The single address every DNS name resolves at (Caddy's edge)."""
for host in self.hosts.values():
if "dns_edge" in host:
return str(host["dns_edge"])
raise SystemExit(f"{self.origin}: no host declares dns_edge")
def service(self, name: str) -> dict | None:
for svc in self.services:
if svc["name"] == name:
return svc
return None
def unit_dirs(self, svc: dict[str, Any], host: str | None = None) -> list[str]:
"""The repo dirs holding this service's unit files, for one host."""
dirs = svc.get("unit_dir")
if not dirs:
return []
if isinstance(dirs, dict):
if host is not None:
return [dirs[host]] if host in dirs else []
return [dirs[h] for h in svc["hosts"] if h in dirs]
return [dirs]
def owns_unit(self, svc: dict[str, Any], host: str, unit: str, directory: str) -> bool:
return unit in svc.get("units", []) and directory in self.unit_dirs(svc, host)
# --------------------------------------------------------------------------
# repo facts
# --------------------------------------------------------------------------
@lru_cache(maxsize=1)
def tracked_files() -> tuple[str, ...]:
# Tracked + untracked (a check that only saw HEAD would ignore the very
# files a work-in-progress commit is adding).
out = subprocess.run(
["git", "-C", str(REPO_ROOT), "ls-files", "--cached", "--others",
"--exclude-standard"],
text=True, stdout=subprocess.PIPE, check=True)
return tuple(line for line in out.stdout.splitlines() if line)
def _network_values(container: Path) -> list[str]:
"""Every `Network=` value a quadlet unit declares (comma lists included)."""
values: list[str] = []
try:
text = container.read_text()
except OSError:
return values
for line in text.splitlines():
if line.startswith("Network="):
values.extend(v.strip() for v in line[len("Network="):].split(",") if v.strip())
return values
class Graph:
"""The unit-file coupling, read out of the tree: netns + netns-net users.
Quadlet installs every user unit flat, so `Network=gluetun.container`
names a unit by name, not by path; the same host is the scope, which is
also the deployment scope.
"""
def __init__(self, hosts: list[str]) -> None:
self.units: dict[tuple[str, str], str] = {} # (host, unit) -> path
self.networks: dict[tuple[str, str], str] = {} # (host, net) -> path
self.netns_children: dict[tuple[str, str], set[tuple[str, str]]] = {}
self.network_users: dict[tuple[str, str], set[tuple[str, str]]] = {}
for path in tracked_files():
parts = PurePosixPath(path).parts
if not parts or parts[0] not in hosts or is_trash(path):
continue
host, directory, name = parts[0], str(PurePosixPath(*parts[1:-1])), parts[-1]
if name.endswith(".container"):
self.units[(host, name[: -len(".container")])] = path
elif name.endswith(".network"):
self.networks[(host, name[: -len(".network")])] = path
for (host, unit), path in self.units.items():
for value in _network_values(REPO_ROOT / path):
if value.endswith(".container"):
parent = (host, value[: -len(".container")])
self.netns_children.setdefault(parent, set()).add((host, unit))
elif value.endswith(".network"):
net = (host, value[: -len(".network")])
self.network_users.setdefault(net, set()).add((host, unit))
def rides(self, start: list[tuple[str, str]]) -> set[tuple[str, str]]:
"""The transitive netns closure: a unit, everything riding it, and
everything riding those (a netns inside a netns)."""
found: set[tuple[str, str]] = set()
frontier = list(start)
while frontier:
node = frontier.pop()
if node in found:
continue
found.add(node)
frontier.extend(self.netns_children.get(node, ()))
return found
@lru_cache(maxsize=8)
def graph_for(hosts: tuple[str, ...]) -> Graph:
return Graph(list(hosts))
# --------------------------------------------------------------------------
# selection
# --------------------------------------------------------------------------
def _owner_index(inv: Inventory) -> dict[tuple[str, str, int]]:
"""(host, unit) -> service index, for unit-file ownership.
A quadlet unit name is unique per host (``check units`` enforces one
claim), so the coupling closure can resolve a riding unit that lives in
a DIFFERENT directory than the changed file — gluetun's netns children
are the point.
"""
index: dict[tuple[str, str], int] = {}
for svc in inv.services:
for host in svc["hosts"]:
for unit in svc.get("units", []):
key = (host, unit)
if key in index and index[key] != svc["_index"]:
raise SystemExit(
f"{inv.origin}: unit {unit!r} on {host} claimed by "
f"{inv.services[index[key]]['name']} and {svc['name']}"
)
index[key] = svc["_index"]
return index
def select(inv: Inventory, host: str, paths: list[str]) -> list[str]:
"""Canonical-order service names the changed paths select for one host."""
if host not in inv.hosts:
raise SystemExit(f"{inv.origin}: no host {host!r}")
graph = graph_for(tuple(inv.hosts))
owners = _owner_index(inv)
hits: set[int] = set()
def add(index: int) -> None:
hits.add(index)
for raw in paths:
path = raw.replace("\\", "/")
parts = PurePosixPath(path).parts
if not parts or is_trash(path) or path in NON_RUNTIME_PATHS:
continue
if PurePosixPath(path).name in NON_RUNTIME_NAMES:
continue
head = parts[0]
if head in NON_RUNTIME_ROOTS:
continue
# A host's directory is the scope: a frankbook-pro file can never
# select a dankserver service.
scoped_host = head if head in inv.hosts else None
if scoped_host is not None and scoped_host != host:
continue
name = parts[-1]
directory = str(PurePosixPath(*parts[1:-1])) if len(parts) > 1 else ""
if scoped_host and name.endswith(UNIT_SUFFIXES):
stem = name.rsplit(".", 1)[0]
key = (scoped_host, stem)
if name.endswith(".network") and key in graph.networks:
users = graph.network_users.get(key, set())
for unit in graph.rides(sorted(users)):
index = owners.get(unit)
if index is not None:
add(index)
elif key in graph.units:
for unit in graph.rides([key]):
index = owners.get(unit)
if index is not None:
add(index)
for index in list(hits):
for dependent in inv.services[index].get("dependents", []):
for svc in inv.named(host, dependent):
add(svc["_index"])
for svc in inv.on(host):
if path in svc.get("files", []):
add(svc["_index"])
if path in svc.get("retires", []):
add(svc["_index"])
for directory_rule in svc.get("dirs", []):
if path == directory_rule or path.startswith(directory_rule + "/"):
add(svc["_index"])
return [svc["name"] for svc in sorted((inv.services[i] for i in hits),
key=lambda s: s["_index"])]
def seams_for(inv: Inventory, host: str, paths: list[str]) -> list[str]:
"""The Bats seams the selected services require, deduplicated, in order."""
seams: list[str] = []
for name in select(inv, host, paths):
for svc in inv.named(host, name):
for seam in svc.get("tests", []):
if seam not in seams:
seams.append(seam)
if seams and "tests/deployment_scoping.bats" not in seams:
seams.append("tests/deployment_scoping.bats")
return seams
# --------------------------------------------------------------------------
# renderers + facts
# --------------------------------------------------------------------------
def render_pihole_hosts(inv: Inventory) -> str:
edge = inv.edge_ip()
lines = [
"# A records for the flat franksdank.cloud scheme (dnsmasq addn-hosts",
"# format: <IP> <name>). RENDERED — do not edit by hand:",
"#",
"# scripts/service-map.py render pihole-hosts > dankclaw/pihole/addn-hosts",
"#",
"# The source of truth is the `dns:` field of shared/services.yml; the",
"# drift gate in tests/service_inventory.bats fails when this file and",
"# the inventory disagree. Caddy is consolidated on Dankclaw, so every",
"# name — including the services running on dankserver and",
"# frankbook-pro — resolves at the Dankclaw edge.",
"#",
"# dnsmasq strips ONLY whole-line comments in addn-hosts: a trailing",
"# comment on a data line becomes a HOSTNAME token. No inline notes",
"# here; the service attribution lives in the inventory.",
"",
]
for svc in inv.services:
for name in svc.get("dns", []):
lines.append(f"{edge} {name}")
return "\n".join(lines) + "\n"
def scrape_facts(inv: Inventory) -> list[dict[str, str]]:
facts = []
for svc in inv.services:
for job in svc.get("scrape", []):
facts.append({"job": job["job"], "target": job["target"],
"metrics_path": job.get("metrics_path", "/")})
return facts
def prometheus_facts(path: Path) -> list[dict[str, str]]:
doc = yaml.safe_load(path.read_text())
facts = []
for job in doc.get("scrape_configs", []):
for sc in job.get("static_configs", []):
for target in sc.get("targets", []):
facts.append({"job": job["job_name"], "target": target,
"metrics_path": job.get("metrics_path", "/")})
return facts
def probe_facts(inv: Inventory) -> list[str]:
facts: list[str] = []
for svc in inv.services:
probe = svc.get("probe")
if isinstance(probe, dict):
facts.extend(str(v) for v in probe.values())
elif probe:
facts.append(str(probe))
return facts
def homepage_probes(path: Path) -> list[str]:
return sorted({s["probe"] for s in json.loads(path.read_text()) if s.get("probe")})
# --------------------------------------------------------------------------
# checks
# --------------------------------------------------------------------------
def check_ownership(inv: Inventory, graph: Graph) -> list[str]:
"""Every repo file belongs to a service, a rule, or the unowned list."""
problems: list[str] = []
exact: set[str] = set()
directories: set[str] = set()
for svc in inv.services:
for host in svc["hosts"]:
for directory in inv.unit_dirs(svc, host):
for unit in svc.get("units", []):
exact.add(f"{directory}/{unit}.container")
for field in ("files", "retires"):
exact.update(svc.get(field, []))
directories.update(svc.get("dirs", []))
# A shared network file is claimed by the units that ride it; the unit
# graph is the referrer, and check_units proves every reference resolves.
exact.update(graph.networks.values())
def is_claimed(path: str) -> bool:
if path in exact:
return True
first = PurePosixPath(path).parts[0]
if first in NON_RUNTIME_ROOTS or path in NON_RUNTIME_PATHS or is_trash(path):
return True
if PurePosixPath(path).name in NON_RUNTIME_NAMES:
return True
for owned in directories | set(inv.unowned):
if path == owned or path.startswith(owned + "/"):
return True
return False
for path in tracked_files():
if not is_claimed(path):
problems.append(f"file belongs to no service and is not in unowned: {path}")
# The reverse direction: an unowned entry that names nothing is stale.
for directory in inv.unowned:
if not any(p == directory or p.startswith(directory + "/") for p in tracked_files()):
problems.append(f"unowned entry names no tracked file: {directory}")
# Every inventory file/dir claim names a tracked file.
for svc in inv.services:
# `retires` names a file whose DELETION selects the service, so by
# definition it is not in the tree.
for field in ("files", "dirs"):
for entry in svc.get(field, []):
if entry not in tracked_files() and not any(
p.startswith(entry + "/") for p in tracked_files()):
problems.append(f"{svc['name']}: {field} entry names no tracked file: {entry}")
del graph
return problems
def check_units(inv: Inventory, graph: Graph) -> list[str]:
"""Every unit file has exactly one owner; every unit reference resolves."""
problems: list[str] = []
owners: dict[str, list[str]] = {}
for svc in inv.services:
for host in svc["hosts"]:
for directory in inv.unit_dirs(svc, host):
for unit in svc.get("units", []):
path = f"{directory}/{unit}.container"
owners.setdefault(path, []).append(f"{svc['name']}@{host}")
if path not in tracked_files():
problems.append(f"{svc['name']}: unit file is not tracked: {path}")
for path in tracked_files():
if not path.endswith(".container") or is_trash(path):
continue
found = owners.get(path, [])
if not found:
problems.append(f"unit file claimed by no service: {path}")
elif len(found) > 1:
problems.append(f"unit file claimed by {len(found)} services: {path}: {found}")
# Root units (scripts/*.service) are claimed by exactly one `files:` list.
for path in tracked_files():
if not (path.startswith("scripts/") and path.endswith(".service")) or is_trash(path):
continue
found = [s["name"] for s in inv.services if path in s.get("files", [])]
if not found:
problems.append(f"root unit claimed by no service: {path}")
elif len(found) > 1:
problems.append(f"root unit claimed by {found}: {path}")
# Every Network= reference in a unit file names a file on the same host.
for (host, unit), path in graph.units.items():
for value in _network_values(REPO_ROOT / path):
if value.endswith(".container"):
target = (host, value[: -len(".container")])
if target not in graph.units:
problems.append(f"{path}: Network={value} names no unit on {host}")
elif value.endswith(".network"):
target = (host, value[: -len(".network")])
if target not in graph.networks:
problems.append(f"{path}: Network={value} names no network on {host}")
return problems
def check_config(inv: Inventory) -> list[str]:
problems: list[str] = []
for svc in inv.services:
label = f"{'+'.join(svc['hosts'])}/{svc['name']}"
for key in REQUIRED_KEYS:
if key not in svc:
problems.append(f"{label}: missing required key {key}")
if svc.get("kind") not in KINDS:
problems.append(f"{label}: kind {svc.get('kind')!r} not one of {sorted(KINDS)}")
unknown = set(svc.get("hosts", [])) - set(inv.hosts)
if unknown:
problems.append(f"{label}: undeclared hosts: {sorted(unknown)}")
if svc.get("units") and not svc.get("unit_dir"):
problems.append(f"{label}: units declared without unit_dir")
if svc.get("deployed_by") and not inv.any_named(svc["deployed_by"]):
problems.append(f"{label}: deployed_by names no service: {svc['deployed_by']}")
if (svc.get("health") and svc.get("wait") is None) or \
(svc.get("wait") is not None and not svc.get("health")):
problems.append(f"{label}: health and wait are a pair")
for seam in svc.get("tests", []):
if seam not in tracked_files():
problems.append(f"{label}: seam does not exist: {seam}")
for name in svc.get("dns", []):
if not str(name).replace(".", "").replace("-", "").isalnum():
problems.append(f"{label}: dns name is not a plain hostname: {name}")
if svc.get("kind") == "root-unit" and svc.get("unit_dir"):
problems.append(f"{label}: a root unit lives in scripts/, not unit_dir")
seen: set[str] = set()
for svc in inv.services:
key = (svc["name"], tuple(svc["hosts"]))
if key in seen:
problems.append(f"duplicate inventory entry: {svc['name']} on {svc['hosts']}")
seen.add(key)
return problems
# --- node-exporter quadlet template (rank 6) --------------------------------
# Parallel per-host copies kept in step by a comment drift apart; the
# template is the single source, and `check` drift-gates every tracked
# rendering against it.
NODE_EXPORTER_TEMPLATE = """\
# Quadlet unit: node-exporter (rootless, boot-persistent) — RENDERED ARTIFACT.
# Regenerate: python3 scripts/service-map.py render node-exporter-unit --host {host}
# Source: shared/services.yml (entry node-exporter) + this template.
#
# Install path on {host}: ~/.config/containers/systemd/node-exporter.container
# Unit name: node-exporter.service (container name: node-exporter)
#
# Host metrics source for Prometheus. Replaces the compose service of the same
# name (the "Observability" block): Dankclaw moved to this quadlet on
# 2026-09-11, Dankserver on 2026-09-16 — its compose-only copy died at a reboot
# and stayed dead (rootless Podman does not restart bare containers after the
# explicit stop shutdown performs), dead for a day and a half before anyone
# noticed. One owner per container name; no compose file remains anywhere.
#
# HOST NAMESPACE. It must run in the host network namespace to report the
# host's own interfaces/connections, and --pid=host for the processes and
# cgroups collectors. "localhost" inside the container is therefore the host's
# loopback, which is what the homepage status timer probes.
#
# LISTEN ADDRESSES. Exactly two: loopback for the homepage status probe, and
# the LAN address, because Prometheus — which runs in a container on
# dankclaw_default — scrapes http://{lan}:{port}/metrics
# (shared/prometheus/prometheus.yml, job "node-{host}"). node_exporter has no
# authentication, so the wildcard bind stays off the table; this is the
# pairing that removes the unauthenticated observability surface on every
# host, and both consumers are accounted for.
#
# MOUNTS. The container reads the host's /proc, /sys and / as /host/*
# read-only and is pointed at them with --path.*; it writes nothing anywhere.
[Unit]
Description=Prometheus node exporter ({title} host metrics)
After=network-online.target
Wants=network-online.target
[Container]
ContainerName=node-exporter
Image=docker.io/prom/node-exporter:latest
Network=host
PodmanArgs=--pid=host
Exec=--path.procfs=/host/proc --path.sysfs=/host/sys --path.rootfs=/rootfs --web.listen-address=127.0.0.1:{port} --web.listen-address={lan}:{port}
Volume=/proc:/host/proc:ro
Volume=/sys:/host/sys:ro
Volume=/:/rootfs:ro
HealthCmd=wget --spider -q http://localhost:{port}/metrics
HealthInterval=30s
HealthTimeout=10s
HealthRetries=3
HealthStartPeriod=30s
[Service]
Restart=always
RestartSec=10
[Install]
WantedBy=default.target
"""
def render_node_exporter_unit(inv: Inventory, host: str) -> str:
"""The node-exporter quadlet for one host: shared template, per-host
title, LAN address from the hosts map, port from the scrape job."""
svc = inv.service("node-exporter")
if svc is None:
raise SystemExit("inventory has no node-exporter entry")
job = f"node-{host}"
port = next((str(e["target"]).rsplit(":", 1)[1] for e in svc.get("scrape", [])
if e.get("job") == job), None)
if port is None:
raise SystemExit(f"node-exporter scrape declares no job {job}")
lan = inv.hosts[host]["address"]
return NODE_EXPORTER_TEMPLATE.format(host=host, title=host.capitalize(),
lan=lan, port=port)
def check_drift(inv: Inventory) -> list[str]:
problems: list[str] = []
rendered = render_pihole_hosts(inv)
hosts_file = REPO_ROOT / "dankclaw" / "pihole" / "addn-hosts"
if hosts_file.read_text() != rendered:
problems.append("dankclaw/pihole/addn-hosts is not the render of the inventory "
"(scripts/service-map.py render pihole-hosts > dankclaw/pihole/addn-hosts)")
want = sorted(json.dumps(f, sort_keys=True) for f in scrape_facts(inv))
have = sorted(json.dumps(f, sort_keys=True) for f in
prometheus_facts(REPO_ROOT / "shared" / "prometheus" / "prometheus.yml"))
for extra in [f for f in have if f not in want]:
problems.append(f"prometheus scrapes a target the inventory does not declare: {extra}")
for missing in [f for f in want if f not in have]:
problems.append(f"inventory declares a scrape prometheus.yml lacks: {missing}")
svc = inv.service("node-exporter")
for host in (svc or {}).get("hosts", []):
for directory in inv.unit_dirs(svc, host):
path = REPO_ROOT / directory / "node-exporter.container"
if path.exists() and path.read_text() != render_node_exporter_unit(inv, host):
problems.append(
f"{directory}/node-exporter.container is not the render of the "
f"template (scripts/service-map.py render node-exporter-unit --host {host})")
probes_want = sorted(set(probe_facts(inv)))
probes_have = homepage_probes(REPO_ROOT / "shared" / "homepage" / "services.json")
for extra in [p for p in probes_have if p not in probes_want]:
problems.append(f"homepage probes a target the inventory does not declare: {extra}")
for missing in [p for p in probes_want if p not in probes_have]:
problems.append(f"inventory declares a probe homepage does not list: {missing}")
return problems
CHECKS = {
"config": lambda inv, graph: check_config(inv),
"units": check_units,
"ownership": check_ownership,
"drift": lambda inv, graph: check_drift(inv),
}
def check(inv: Inventory, names: list[str]) -> list[str]:
graph = graph_for(tuple(inv.hosts))
problems: list[str] = []
for name in names or list(CHECKS):
problems.extend(CHECKS[name](inv, graph))
return problems
# --------------------------------------------------------------------------
# CLI
# --------------------------------------------------------------------------
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--inventory", type=Path, default=DEFAULT_INVENTORY)
sub = parser.add_subparsers(dest="command", required=True)
sub.add_parser("hosts")
p = sub.add_parser("services")
p.add_argument("--host")
p = sub.add_parser("allows")
p.add_argument("--host", required=True)
p.add_argument("--service", required=True)
p = sub.add_parser("plan")
p.add_argument("--host", required=True)
p.add_argument("paths", nargs="+")
p = sub.add_parser("seams")
p.add_argument("--host")
p.add_argument("paths", nargs="+")
p = sub.add_parser("owner")
p.add_argument("--host", required=True)
p.add_argument("path")
p = sub.add_parser("units")
p.add_argument("--host", required=True)
p.add_argument("--service", required=True)
p = sub.add_parser("stack")
p.add_argument("--host", required=True)
p.add_argument("--service", required=True)
p = sub.add_parser("get")
p.add_argument("--host")
p.add_argument("--service", required=True)
p.add_argument("field")
p = sub.add_parser("render")
p.add_argument("artifact", choices=("pihole-hosts", "node-exporter-unit"))
p.add_argument("--host", choices=list(Hosts) if False else None)
p = sub.add_parser("check")
p.add_argument("names", nargs="*")
args = parser.parse_args(argv)
inv = Inventory.load(args.inventory)
if args.command == "hosts":
print("\n".join(inv.hosts))
elif args.command == "services":
services = inv.on(args.host) if args.host else inv.services
seen: list[str] = []
for svc in services:
if svc["name"] not in seen:
seen.append(svc["name"])
print("\n".join(seen))
elif args.command == "allows":
return 0 if inv.named(args.host, args.service) else 1
elif args.command == "plan":
print("\n".join(select(inv, args.host, args.paths)))
elif args.command == "seams":
if args.host is not None:
seams = seams_for(inv, args.host, args.paths)
else:
# No host: the union across every host (the dev machine's
# pre-commit mapper has no deployment target).
seams = []
for host in inv.hosts:
for seam in seams_for(inv, host, args.paths):
if seam not in seams:
seams.append(seam)
print("\n".join(seams))
elif args.command == "owner":
found = select(inv, args.host, [args.path])
print(found[-1] if found else "")
elif args.command == "units":
matches = inv.named(args.host, args.service)
if not matches:
print(f"no service {args.service} on {args.host}", file=sys.stderr)
return 2
for svc in matches:
for directory in inv.unit_dirs(svc, args.host):
for unit in svc.get("units", []):
print(f"{unit} {directory}")
elif args.command == "stack":
matches = inv.named(args.host, args.service)
if not matches:
print(f"no service {args.service} on {args.host}", file=sys.stderr)
return 2
owner = matches[0].get("deployed_by") or args.service
print(owner)
for svc in inv.on(args.host):
if svc.get("deployed_by") == owner and svc["name"] != owner:
print(svc["name"])
elif args.command == "get":
matches = inv.on(args.host) if args.host else inv.services
matches = [s for s in matches if s["name"] == args.service]
if not matches:
print(f"no service {args.service}" + (f" on {args.host}" if args.host else ""),
file=sys.stderr)
return 2
values = [s.get(args.field) for s in matches if args.field in s]
print(json.dumps(values[0] if len(values) == 1 else values))
elif args.command == "render":
if args.artifact == "pihole-hosts":
sys.stdout.write(render_pihole_hosts(inv))
else:
if not args.host:
print("render node-exporter-unit requires --host", file=sys.stderr)
return 2
sys.stdout.write(render_node_exporter_unit(inv, args.host))
elif args.command == "check":
problems = check(inv, args.names)
if problems:
for problem in problems:
print(f"FAIL: {problem}", file=sys.stderr)
return 1
print("OK: the service inventory is sound (schema, units, ownership, drift)")
return 0
if __name__ == "__main__":
sys.exit(main())
+3 -3
View File
@@ -293,12 +293,12 @@ def main():
ddt, dsh = meta[f"{pre0}.mlp.experts.0.down_proj.weight"][3:5]
gsd, gssh = meta[f"{pre0}.mlp.experts.0.gate_proj.weight_scale_inv"][3:5]
dsd, dssh = meta[f"{pre0}.mlp.experts.0.down_proj.weight_scale_inv"][3:5]
scale_law = (gdt == F8 and ddt == F8 and gsd == BF16 and dsd == BF16
scale_contract = (gdt == F8 and ddt == F8 and gsd == BF16 and dsd == BF16
and list(gssh) == [(gsh[0] + 127) // 128, (gsh[1] + 127) // 128]
and list(dssh) == [(dsh[0] + 127) // 128, (dsh[1] + 127) // 128])
if not scale_law:
if not scale_contract:
log(f"ERROR: unexpected expert shapes: gate {gdt}{gsh} scale {gsd}{gssh} "
f"down {ddt}{dsh} scale {dsd}{dssh} (scale law 128x128)")
f"down {ddt}{dsh} scale {dsd}{dssh} (fp8 block scales cover 128x128 tiles)")
return 2
shapes = {"ffn2_h": (gsh[0] * 2, gsh[1]), "ffn": (dsh[1],),
"gs": tuple(gssh), "ds": tuple(dssh)}
+57 -190
View File
@@ -62,234 +62,101 @@ declare -a TESTS=()
add_test() {
local test="$1"
local existing
MAPPED=1
for existing in "${TESTS[@]:-}"; do
[ "$existing" != "$test" ] || return 0
done
TESTS+=("$test")
}
# The path->test mapping has two layers:
# 1. cross-cutting seams that no single service owns (compose validity,
# env consistency, the deployer itself, docs, syntax);
# 2. the service inventory (shared/services.yml via service-map.py
# "seams"): every service the path selects on ANY host contributes its
# declared tests. Service seams are the services' own — the former
# shotgun fallbacks (every dankclaw quadlet unit ran the open-webui,
# pihole, and searxng seams) are retired; the inventory names each
# service's real seams, and adding a service is a data edit.
for path in "${CHANGED[@]}"; do
MAPPED=0
case "$path" in
tests/*.bats)
add_test "$path"
continue
;;
tests/helpers.bash|tests/run.sh)
die "$path is shared test infrastructure; run ./tests/run.sh explicitly"
;;
.githooks/*)
add_test tests/changed_tests.bats
;;
dankserver/compose.yaml|dankclaw/compose.yaml)
add_test tests/compose_validity.bats
add_test tests/env_consistency.bats
add_test tests/deployment_scoping.bats
;;
dankserver/Caddyfile|dankclaw/Caddyfile)
add_test tests/caddyfile.bats
add_test tests/deployment_scoping.bats
;;
tests/e2e/*)
add_test tests/e2e_scaffold.bats
continue
;;
.githooks/*)
add_test tests/changed_tests.bats
continue
;;
# Archive material and host-level facts: a path under any
# .AGENTTRASH/ directory is retired-by-policy (the planner plans
# nothing from it), udev rules are applied by manual sudo install,
# reports are frozen artifacts, and ignore-lists change nothing at
# runtime. They select no focused test; without this case they would
# trip the mapping gate below.
.AGENTTRASH/*|*/.AGENTTRASH/*|dankclaw/udev/*|report/*|.gitignore|.env.example)
continue
;;
esac
# Cross-cutting seams.
case "$path" in
dankserver/.env.example|dankclaw/.env.example)
add_test tests/env_consistency.bats
;;
shared/homepage/*)
add_test tests/homepage_caddy.bats
add_test tests/homepage_config.bats
add_test tests/homepage_deploy.bats
add_test tests/homepage_pages.bats
add_test tests/homepage_checker.bats
add_test tests/deployment_scoping.bats
;;
shared/searxng/*)
add_test tests/searxng_limiter.bats
add_test tests/searxng_quadlet.bats
add_test tests/deployment_scoping.bats
;;
# The five observability containers left compose for Quadlet units on
# 2026-09-11. Their seam pins boot persistence and the loopback-only
# publish posture that keeps them off every host interface. This case
# precedes the dankclaw/*/quadlet/*.container fallback below, which
# would otherwise swallow it into unrelated seams.
dankclaw/observability/*)
add_test tests/observability_quadlet.bats
dankserver/Caddyfile|dankclaw/Caddyfile)
add_test tests/caddyfile.bats
add_test tests/deployment_scoping.bats
add_test tests/docs_completeness.bats
;;
# Caddy left compose.yaml on 2026-08-26 for a quadlet; caddyfile.bats
# now asserts against the quadlet itself. Route every change under
# dankclaw/caddy/ to both seams (this block must precede the
# dankclaw/*.container glob, which would otherwise swallow the unit).
dankclaw/caddy/*)
add_test tests/caddy_quadlet.bats
add_test tests/caddyfile.bats
add_test tests/docs_completeness.bats
add_test tests/deployment_scoping.bats
;;
dankclaw/*.container|dankclaw/*.network)
add_test tests/open_webui_quadlet.bats
add_test tests/pihole.bats
add_test tests/searxng_quadlet.bats
add_test tests/deployment_scoping.bats
;;
# Anubis is a quadlet + Caddyfile pair: the chain (public listener ->
# anubis:8923 -> internal :8016 listener) spans both, so any change
# under the service dir must re-run the caddyfile seam too. This case
# precedes the dankclaw/*/quadlet/*.container fallback below, which
# would otherwise swallow it into unrelated seams.
dankclaw/anubis/*)
add_test tests/anubis.bats
add_test tests/caddyfile.bats
add_test tests/homepage_caddy.bats
add_test tests/deployment_scoping.bats
;;
# frank-dragoon is a rootless quadlet again since 2026-09-16 (the
# halogen-flash-server restore). Its unit carries the fleet's GPU and
# podman posture, so it gets its own seam rather than the generic
# fallback below.
dankclaw/frank-dragoon/*)
add_test tests/frank_dragoon_unit.bats
add_test tests/deployment_scoping.bats
add_test tests/docs_completeness.bats
;;
# Quadlets live under dankclaw/<svc>/quadlet/ (reorganized out of the
# top-level dankclaw/*.container globs above). Map the whole subtree so
# changes to the pihole, open-webui, searxng, and gitea quadlets hit
# their seam tests. The pihole-specific block must come first: shell
# case matches the first pattern, and dankclaw/pihole/quadlet/*.container
# also satisfies the dankclaw/*/quadlet/*.container fallback.
dankclaw/pihole/*)
add_test tests/pihole.bats
add_test tests/docs_completeness.bats
add_test tests/deployment_scoping.bats
;;
dankclaw/*/quadlet/*.container|dankclaw/*/quadlet/*.network)
add_test tests/open_webui_quadlet.bats
add_test tests/pihole.bats
add_test tests/searxng_quadlet.bats
add_test tests/docs_completeness.bats
add_test tests/deployment_scoping.bats
;;
scripts/deploy.sh|scripts/deploy-remote.sh|scripts/deploy-plan.py|scripts/deploy-*-quadlet.sh)
add_test tests/deployment_flow.bats
add_test tests/deployment_scoping.bats
add_test tests/scripts_syntax.bats
;;
scripts/service-map.py|shared/services.yml)
add_test tests/service_inventory.bats
add_test tests/deployment_scoping.bats
add_test tests/scripts_syntax.bats
;;
scripts/lib/*.sh)
add_test tests/llm_gates.bats
add_test tests/scripts_syntax.bats
;;
serverAppConfigs-docs/*|README.md|AGENTS.md)
add_test tests/docs_completeness.bats
add_test tests/deployment_scoping.bats
;;
scripts/mount-entropy.sh)
add_test tests/mount_entropy.bats
;;
# Archive material: a path under any .AGENTTRASH/ directory is
# retired-by-policy (deploy-plan.py plans nothing from it), so it
# selects no focused test. It still needs its own case: without one
# the wrapper refuses retirement commits, whose trash moves show up
# as additions, with "no focused-test mapping".
.AGENTTRASH/*|*/.AGENTTRASH/*)
;;
# frank-archon is llama.cpp (the unsloth b11030-mix prebuilt) on the
# two R9700s as a ROOT unit — the 2026-09-26 swap that replaced the
# Lucebox DS4.1 quadlet on :7770. Its env, unit and serve script all
# live in scripts/, which is why this case has to precede the generic
# scripts/*.service and scripts/* fallbacks below.
scripts/frank-archon.*|scripts/serve-frank-archon.sh)
add_test tests/frank_archon_unit.bats
add_test tests/llm_units.bats
add_test tests/deployment_scoping.bats
add_test tests/scripts_syntax.bats
;;
# frank-dragoon's chat template lives in scripts/ (moved there on
# 2026-09-26 when its old home was retired): a template edit changes
# what the live endpoint renders, so it selects the large unit seam.
scripts/qwen-sharp-v22.4.0.jinja)
add_test tests/frank_dragoon_unit.bats
;;
# frank-zealot is llama.cpp on the 3090 as a ROOT unit: a native CUDA
# binary, so no generated quadlet is involved. Its env, unit and serve
# script all live in scripts/, which is why this case has to precede
# the generic scripts/*.service and scripts/* fallbacks below.
scripts/frank-zealot.*|scripts/serve-frank-zealot.sh)
add_test tests/frank_zealot_unit.bats
add_test tests/llm_units.bats
add_test tests/deployment_scoping.bats
old/scripts/*)
add_test tests/scripts_syntax.bats
;;
scripts/*.service)
add_test tests/llm_units.bats
add_test tests/scripts_syntax.bats
add_test tests/deployment_scoping.bats
;;
scripts/*)
add_test tests/scripts_syntax.bats
;;
serverAppConfigs-docs/*)
add_test tests/docs_completeness.bats
add_test tests/deployment_scoping.bats
;;
dankclaw/hermes/*)
add_test tests/docs_completeness.bats
;;
README.md|AGENTS.md)
# Project guidance changes can affect the documented deployment
# contract; validate the docs vault and scoped deployment seams.
add_test tests/docs_completeness.bats
add_test tests/deployment_scoping.bats
;;
dankclaw/comfyui/*)
add_test tests/docs_completeness.bats
add_test tests/deployment_scoping.bats
;;
# loki and alloy images are built from this repo's Containerfiles; the
# loki one carries the healthcheck command (see its comment), so both
# seams read it.
dankclaw/loki/*|dankclaw/alloy/*)
add_test tests/observability_quadlet.bats
add_test tests/caddyfile.bats
add_test tests/deployment_scoping.bats
;;
dankclaw/udev/*)
# Host-level udev rules are applied by manual sudo install + udevadm
# trigger, not dispatched as a service; nothing here is bats-testable.
;;
.AGENTTRASH/*)
# Trash moves are not testable; skip.
;;
.gitignore|.env.example)
# Ignore/ignore-list changes change nothing at runtime; not testable.
;;
old/scripts/*|old/scripts/**)
add_test tests/scripts_syntax.bats
;;
# Cutover-era dankserver quadlet units (romm trio, jellyfin, calibre-web,
# copyparty, plex, tautulli): the seam checks boot-persistence +
# v1-generator contract; deploy-flow seams guard the deployer routing.
dankserver/romm/*|dankserver/jellyfin/*|dankserver/calibre-web/*|dankserver/copyparty/*|dankserver/plex/*|dankserver/tautulli/*)
add_test tests/dankserver_quadlet.bats
add_test tests/deployment_flow.bats
add_test tests/deployment_scoping.bats
;;
# FrankBook-Pro units live inside the podman machine VM; the seam
# pins boot persistence, the gluetun netns invariant, and the
# plan-locked volume layout. Deployer changes route here too.
frankbook-pro/*)
add_test tests/frankbook_quadlet.bats
add_test tests/deployment_flow.bats
add_test tests/deployment_scoping.bats
;;
# Seerr unit on dankclaw: boot persistence, loopback-only port
# posture, and the deployer routing that installs it.
dankclaw/seerr/*)
add_test tests/seerr_quadlet.bats
add_test tests/deployment_flow.bats
add_test tests/deployment_scoping.bats
;;
report/*)
# Frozen audit reports are point-in-time artifacts; not testable.
;;
*)
die "no focused-test mapping for '$path'; add one before committing"
;;
esac
# The inventory's declared seams for every service this path selects.
while IFS= read -r seam; do
[ -n "$seam" ] && add_test "$seam"
done < <(python3 "$REPO_ROOT/scripts/service-map.py" seams -- "$path" 2>/dev/null || true)
# Mapping gate: a path that selected no test at all — no service seam,
# no cross-cutting rule — is an unmapped area of the repo; refuse to
# commit it silently (the old catch-all `die`).
if [ "$MAPPED" -eq 0 ]; then
die "no focused-test mapping for '$path'; map it in the inventory (shared/services.yml) before committing"
fi
done
# Mapped-but-untestable paths (udev rules, trash moves) legitimately produce an
@@ -0,0 +1,100 @@
---
type: reference
title: Service Inventory
description: shared/services.yml is the one inventory of deployable services; scripts/service-map.py is its only reader. The planner, both deploy allow-lists, the changed-test seam mapper, the Pi-hole hosts render, the Prometheus scrape facts, and the homepage probes all derive from it.
tags: [deployment, inventory, planner, quadlet, prometheus, pihole, homepage]
timestamp: 2026-10-13T00:00:00-07:00
---
# Service Inventory
Before 2026-10-13 the same facts lived in four hand-maintained lists — the
planner's path rules (`deploy-plan.py`), the `--service` allow-list in
`deploy.sh`, the near-copy in `deploy-remote.sh`, and the changed-test mapper
(`test-changed.sh`) — plus per-fact copies in `prometheus.yml`,
`addn-hosts`, and `services.json`. They drifted on every swap (the planner
still mapped the retired `qwen-sharp` template to frank-dragoon; the allow-list
still listed the *arr stack under Dankserver after it moved to FrankBook-Pro).
Now there is **one inventory and one reader**:
- `shared/services.yml` — the data: every deployable service, its hosts, unit
dir(s), unit files, path claims, tests, scrape/DNS/probe facts, aliases.
- `scripts/service-map.py` — the only reader. CLI subcommands:
`hosts`, `services`, `allows`, `plan`, `seams`, `owner`, `units`, `stack`,
`get`, `render`, `check`.
## 1. Consumers
|Consumer|Call|
|---|---|
|`scripts/deploy-plan.py`|thin wrapper → `plan --host TARGET [paths...]` (keeps the `--repo/--base/--head` and `--paths` CLIs)|
|`scripts/deploy.sh` (`--service` gate)|`allows --host T --service S`|
|`scripts/deploy-remote.sh` (`service_is_allowed`)|same|
|`scripts/test-changed.sh`|`seams [paths...]` — a service's declared `tests` are its seams|
|`dankclaw/pihole/addn-hosts`|`render pihole-hosts` — the tracked file is a rendered artifact|
|`*/observability/quadlet/node-exporter.container`|`render node-exporter-unit --host H` — the MIRROR pair became one template; per-host title, LAN address (hosts map), and port (scrape job) fill it|
|Prometheus scrape jobs, LAN DNS, homepage probes|`check` compares the tracked configs against the inventory's `scrape`/`dns`/`probe` fields (drift gate)|
Selection rules the engine encodes (see `tests/fixtures/service-plan-golden.tsv`,
the pinned characterization of every tracked path × every target):
- **Host scoping** — a host's top-level directory can only select that host's
services. `frankbook-pro/**` plans nothing on Dankserver (the old planner
had a stale Dankserver *arr list).
- **Quadlet coupling graph** — `Network=x.container` rides x's netns;
`Network=y.network` joins a shared network. Changing the parent (or the
network file) selects every rider transitively, in their own directories
(`dankclaw/searxng/quadlet/gluetun.container` → gluetun + searxng + marreta;
`frankbook-pro/gluetun/quadlet/gluetun.container` → the whole arr stack).
- **Stacks** — a service's `stack` members deploy and restart as one handler
(romm trio, hermes trio).
- **Retirement** — removing a service from `services.yml` removes it from the
planner, both allow-lists, and the seam mapper in one data edit; a pending
deploy-state entry for it is auto-triaged (see [[Deploy State Triage]]).
## 2. Integrity gates (`check`)
`python3 scripts/service-map.py check` runs, and
`tests/service_inventory.bats` pins:
- **schema** — required fields per entry, known keys only;
- **units** — every unit file named by an entry exists under its `unit_dir`
on the right host; each unit name is claimed by exactly one service per host;
- **ownership** — every tracked quadlet `.container`/`.network` file is claimed
by exactly one service (trash excluded);
- **paths** — every claimed `files`/`dirs`/`retires` glob matches ≥1 tracked file;
- **compose** — the container names in each host's `compose.yaml` have exactly
one quadlet owner (the [[Quadlet Lifecycle]] two-owners trap);
- **drift** — `addn-hosts` equals its render, each tracked node-exporter unit
equals its per-host render; the Prometheus job set, the DNS names, and the
homepage probe ports equal the inventory's fields.
## 3. Adding, moving, retiring a service
- **Add**: append an entry (name, hosts, kind, unit_dir, units, paths, tests,
…), add the quadlet file, run `check` — `plan`/`allows`/`seams` pick it up
with zero script edits.
- **Move between hosts**: edit `hosts` (and per-host `unit_dir`/`probe` maps).
- **Retire**: delete the entry, move files to `.AGENTTRASH/` (see
[[Testing Policy]] for the seam the mapper applies to trash paths).
Field notes: `alias` carries deployable names that differ from the service
name (`calibre-web` deploys the unit `calibre-server`); `probe` for multi-host
services is a `{host: port}` map; `dependents` names services whose config
embeds a reference to this one (config-only coupling — a `.env` edit selects
both, a unit edit selects the graph closure).
## 4. Known cutover deltas (2026-10-13)
The 17 deltas against the old planner are all in these classes: anubis paths
now select `anubis` (the old planner had no rule at all); cross-host paths
select nothing; `scripts/qwen-sharp-v22.4.0.jinja` (retired template) plans
nothing while `scripts/qwen38-froggeric-v22.5.jinja` (the live frank-archon
template, previously unmapped) plans frank-archon;
`dankserver/romm/romm.container` (top-level quadlet dir the old unit-file rule
missed) now plans the romm trio.
---
Related: [[Deployment Flow]], [[Testing Policy]], [[Quadlet Lifecycle]],
[[Container Network Namespaces]], [[Deploy State Triage]]
@@ -15,10 +15,9 @@ This repository tests its configuration at defined seams. The tests run with `ba
| Seam | File | What it asserts |
|------|------|-----------------|
| Compose validity | `tests/compose_validity.bats` | YAML parses, services have images, no named volumes |
| Env consistency | `tests/env_consistency.bats` | every `${VAR}` is defined in `.env.example`; no real secrets; the live gitea password never appears |
| Caddyfile validity | `tests/caddyfile.bats` | per-host Caddyfile, `.local` convention, balanced braces, basic auth present |
| Docs completeness | `tests/docs_completeness.bats` | every compose service has a docs page; index links every page |
| Docs completeness | `tests/docs_completeness.bats` | every service has a docs page; index links every page; no unresolved vault links |
| Script syntax | `tests/scripts_syntax.bats` | `bash -n` clean, executable bits, shellcheck when available |
| Deployment flow | `tests/deployment_flow.bats`, `tests/deployment_scoping.bats` | scoped dispatcher exists, targets are known, service plans isolate inference, Caddy ownership is guarded, and SSH reachability is checked (skips offline) |
| Quadlet units | `tests/*quadlet*.bats` | every unit is boot-persistent, self-healing, repo-provenanced, secret-free |
@@ -357,11 +357,15 @@ client → TLS edge → Anubis :8923 → outpost decision → internal
routing matches on hostname, verified 2026-09-02).
`internal_host = http://caddy:8017`, all three flows set, and an
Application attached — the two provisioning rules above.
2. **Route flip in ONE commit/deploy**: the vhost becomes
`reverse_proxy anubis:8923` with `header_up X-Real-Ip {remote_host}`, and
inside the `http://:8017` listener add `@<svc> host <vhost>` +
2. **Route flip in ONE commit/deploy**: the vhost body becomes
`import guarded` — the named snippet at the top of the Caddyfile is the
whole pre-auth chain (`reverse_proxy anubis:8923` + `header_up X-Real-Ip
{remote_host}`; rank 6 retired the nine verbatim copies), and inside the
`http://:8017` listener add `@<svc> host <vhost>` +
`handle @<svc> { reverse_proxy <upstream> }`. Anubis guard set, Caddy TLS
edge, `:8017` routing table, and provider list change together.
`tests/caddyfile.bats` pins that the chain exists exactly once as the
`(guarded)` snippet and every guarded vhost imports it.
3. Deploy via the repo deploy flow (commit first; deploy pushes main itself).
**Verification (all three):**
@@ -27,7 +27,7 @@ conversation slot and unquantized KV back.
| Host / device | Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB |
| Endpoint | `http://192.168.0.143:7771` — served model id **`frank-zealot`** (`ALIAS` in the env; the SERVICE keeps the `frank-zealot` name), `/v1` OpenAI surface |
| Runtime | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build `…/llama-master/build-cuda/bin/llama-server` (`GGML_CUDA_FA_ALL_QUANTS=ON`); `qwen35` is registered in the build's own `src/llama-arch.cpp` and `src/models/qwen35.cpp` exists at that commit |
| Unit | root systemd `frank-zealot.service` ([`scripts/frank-zealot.service`](../../scripts/frank-zealot.service)), `User=dank`, `Restart=on-failure`, `LimitMEMLOCK=infinity` |
| Unit | root systemd `frank-zealot.service` ([`scripts/frank-zealot.service`](../../scripts/frank-zealot.service)), `User=dank`, `Restart=always` (raised from `on-failure` 2026-10-13 for fleet parity — exit 0 is a CLEAN stop to systemd, so the weaker policy leaves the compaction hub dark while the unit reports success; `tests/llm_units.bats` gates it), `LimitMEMLOCK=infinity` |
| Serve script | [`scripts/serve-frank-zealot.sh`](../../scripts/serve-frank-zealot.sh), tunables [`scripts/frank-zealot.env`](../../scripts/frank-zealot.env) |
| Weights | `/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf` — HauhauCS **Q6_K**, 7,359,259,008 B, sha256 `c0ba7beb68fd3fe47891bd549486d38dcf62d00817296ea314ad37017f5a4986` |
| Projector | `mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf`, 921,704,448 B, sha256 `05f662501f8bd45607b079723a3e238a4e888fd085a10a53f4057a0e250f6934` — the model is natively multimodal and the Q6_K file carries no vision tensors, so this file IS the vision capability |
+1
View File
@@ -28,6 +28,7 @@ covers both workloads).
- [[Network Architecture]] — Caddy reverse proxy, hostname routing, Gluetun VPN path
- [[Storage Policy]] — bind mounts, data locations, ownership
- [[Deployment Flow]] — service-scoped git deploy, target map, and health checks
- [[Service Inventory]] — shared/services.yml, the one inventory of deployable services, and service-map.py
- [[Testing Policy]] — the test seams and how to run them
- [[E2E Suite]] — the Playwright suite that drives the live guard chain
- [[Container Network Namespaces]] — which local services are addressable from each Podman network namespace on Dankclaw
+598
View File
@@ -0,0 +1,598 @@
# =============================================================================
# services.yml — THE service inventory.
#
# One entry per deployable service, in canonical deploy order (dependencies
# before consumers, the LLM fleet last). Four hand-maintained lists used to
# agree with each other by discipline alone; every one of them now READS this
# file:
#
# scripts/deploy-plan.py which services a changed path selects
# scripts/deploy-remote.sh which services a target accepts (allow-list)
# scripts/test-changed.sh which Bats seams a changed path must re-run
# scripts/service-map.py renderers + drift gates (`check`)
#
# `service-map.py check` is the referee: every quadlet unit file and every
# root unit in the tree is claimed by exactly one entry, every repo file is
# owned by a service or listed in `unowned:` with a reason, every seam exists,
# the rendered Pi-hole hosts file equals the render, and Prometheus' scrape
# targets plus the homepage probes equal the `scrape:` / `probe:` facts.
#
# --- fields ------------------------------------------------------------------
# name the deployer's service word == the systemd unit name for
# quadlet / root-unit / vm-quadlet kinds
# hosts targets that accept it (the deploy-remote.sh allow-list)
# kind quadlet rootless user unit from a .container file
# root-unit systemd system unit symlinked out of scripts/
# vm-quadlet quadlet inside the FrankBook-Pro podman machine
# host a host-level script/timer, no container
# deployed_by this service deploys as part of another deployable's handler
# (the trio/pair shape). It has no dirs/files of its own; the
# owner names the deployable whose handler installs it.
# unit_dir repo dir holding <unit>.container — a scalar, or a per-host map
# when one service carries its unit on two hosts (node-exporter)
# units container names this deployable owns (netns coupling is derived
# from the `Network=` lines in those files, never declared here)
# dirs repo dirs whose every file selects this service
# files exact repo files that select this service (root-unit trios,
# vendored templates, host scripts installed by this handler)
# retires files whose DELETION selects this service (the unit the
# handler must remove before it can claim the ports)
# only Dankserver's media stack still has one)
# health the deployer's readiness URL; `wait` its 2 s poll budget
# scrape Prometheus jobs: {job, target, metrics_path?, labels?}
# dns names this service answers at (rendered into the Pi-hole
# addn-hosts; every vhost resolves at the Dankclaw Caddy edge)
# probe host:port the homepage checker dials (per-host map if on 2 hosts)
# tests the Bats seams a change to this service must re-run
# =============================================================================
hosts:
dankclaw:
address: 192.168.0.143
# Caddy is consolidated on Dankclaw: it fronts every service on every
# host, so every DNS name in the fleet resolves here.
dns_edge: 192.168.0.143
dankserver:
address: 192.168.0.235
frankbook-pro:
address: 192.168.0.112
# Repo files no service owns. Each entry is a fact about the tree, not a
# fallback: the census gate fails on anything unowned that is NOT listed, so
# retirement work shrinks this list and a new stray file cannot slip in.
unowned:
shared/services.yml: the inventory this tool reads
scripts: repo tooling (build, fetch, bench, deploy, test); the deployers'
files are claimed by name in the service entries
autoresearch.sh: bench scratch at the repo root (retirement pending)
echo_prompt.txt: bench fixture at the repo root (retirement pending)
echo_request.json: bench fixture at the repo root (retirement pending)
prose_prompt.txt: bench fixture at the repo root (retirement pending)
prose_request.json: bench fixture at the repo root (retirement pending)
dankclaw/apply_qwen21_patch.py: retired patch script (its target runtime is gone)
dankclaw/comfyui-start.sh: superseded by the comfyui quadlet + Containerfile
dankclaw/udev: host udev rules, applied by manual sudo install + udevadm trigger
tests: the seams themselves
report: frozen point-in-time audit reports
old: retired scripts, reference only
.githooks: the commit hooks that run the seams
services:
# --- Search stack: one netns family, gluetun owns it ---------------------
- name: gluetun
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/searxng/quadlet
units: [gluetun]
tests: [tests/searxng_quadlet.bats]
# The tunnel's WireGuard bind — the homepage card for the tunnel itself.
probe: 127.0.0.1:51820
- name: searxng-valkey
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/searxng/quadlet
units: [searxng-valkey]
# An app-level dependency, not a network one: the deploy handler
# restarts SearXNG after its cache, so the cache's files select both.
dependents: [searxng]
tests: [tests/searxng_quadlet.bats]
probe: 127.0.0.1:6379
- name: searxng
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/searxng/quadlet
units: [searxng]
dirs: [shared/searxng]
tests: [tests/searxng_limiter.bats, tests/searxng_quadlet.bats]
scrape:
- job: services
target: 192.168.0.143:8081
metrics_path: /
dns: [search.franksdank.cloud]
probe: 127.0.0.1:8081
- name: marreta
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/marreta/quadlet
units: [marreta, marreta-browser]
tests: [tests/marreta_quadlet.bats]
dns: [marreta.franksdank.cloud]
probe: 127.0.0.1:8082
# --- DNS ------------------------------------------------------------------
- name: pihole
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/pihole/quadlet
units: [pihole]
dirs: [dankclaw/pihole]
tests: [tests/pihole.bats, tests/docs_completeness.bats]
dns: [pihole.franksdank.cloud]
probe: 192.168.0.143:8083
# --- Chat front end -------------------------------------------------------
- name: open-webui
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/open-webui/quadlet
units: [open-webui]
tests: [tests/open_webui_quadlet.bats, tests/caddyfile.bats]
scrape:
- job: services
target: 192.168.0.143:3005
metrics_path: /
dns: [ai.franksdank.cloud]
probe: 127.0.0.1:3005
# --- Git hosting ----------------------------------------------------------
- name: gitea-postgres
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/gitea/quadlet
units: [gitea-postgres]
deployed_by: gitea-server
tests: [tests/caddyfile.bats, tests/docs_completeness.bats]
probe: 127.0.0.1:5432
- name: gitea-server
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/gitea/quadlet
units: [gitea-server]
tests: [tests/caddyfile.bats, tests/docs_completeness.bats]
scrape:
- job: gitea
target: 192.168.0.143:3000
metrics_path: /metrics
dns: [gitea.franksdank.cloud]
probe: 127.0.0.1:3000
# --- Image generation -----------------------------------------------------
- name: comfyui
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/comfyui/quadlet
units: [comfyui]
dirs: [dankclaw/comfyui]
files: [dankclaw/Containerfile.comfyui]
health: http://127.0.0.1:8188/
wait: 300
tests: [tests/comfyui_rocm.bats, tests/docs_completeness.bats]
scrape:
- job: comfyui
target: comfyui:8188
metrics_path: /
dns: [comfyui.franksdank.cloud]
probe: 127.0.0.1:8188
# --- Requests -------------------------------------------------------------
- name: seerr
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/seerr/quadlet
units: [seerr]
tests: [tests/seerr_quadlet.bats, tests/deployment_flow.bats]
dns: [seerr.franksdank.cloud]
probe: 127.0.0.1:5055
# --- Agent stack: one handler, four containers ----------------------------
- name: hermes
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/hermes/quadlet
units: [hermes, hermes-dashboard, hermes-webui, chrome-devtools-protocol]
dirs: [dankclaw/hermes]
tests: [tests/hermes_webui.bats, tests/docs_completeness.bats]
dns: [hermes.franksdank.cloud]
probe: 127.0.0.1:8787
- name: hermes-dashboard
hosts: [dankclaw]
kind: quadlet
deployed_by: hermes
tests: [tests/hermes_webui.bats, tests/docs_completeness.bats]
dns: [hermesadmin.franksdank.cloud]
probe: 192.168.0.143:9119
- name: hermes-webui
hosts: [dankclaw]
kind: quadlet
deployed_by: hermes
tests: [tests/hermes_webui.bats, tests/docs_completeness.bats]
# --- Observability --------------------------------------------------------
- name: prometheus
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/observability/quadlet
units: [prometheus]
dirs: [shared/prometheus]
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
tests/docs_completeness.bats]
scrape:
- job: prometheus
target: localhost:9090
dns: [prometheus.franksdank.cloud]
probe: 127.0.0.1:9090
- name: grafana
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/observability/quadlet
units: [grafana]
dirs: [shared/grafana]
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
tests/docs_completeness.bats]
scrape:
- job: grafana
target: grafana:3000
metrics_path: /metrics
dns: [grafana.franksdank.cloud]
probe: 127.0.0.1:3002
- name: loki
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/observability/quadlet
units: [loki]
dirs: [dankclaw/loki, shared/loki]
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
tests/docs_completeness.bats]
scrape:
- job: loki
target: loki:3100
metrics_path: /metrics
dns: [loki.franksdank.cloud]
probe: 127.0.0.1:3100
- name: alloy
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/observability/quadlet
units: [alloy]
dirs: [dankclaw/alloy, shared/alloy]
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
tests/docs_completeness.bats]
probe: 127.0.0.1:12345
- name: node-exporter
# One service on two hosts: the same exporter, its own unit per host.
hosts: [dankclaw, dankserver]
kind: quadlet
unit_dir:
dankclaw: dankclaw/observability/quadlet
dankserver: dankserver/observability/quadlet
units: [node-exporter]
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
tests/docs_completeness.bats]
scrape:
- job: node-dankclaw
target: host.containers.internal:9100
- job: node-dankserver
target: 192.168.0.235:9100
probe: 127.0.0.1:9100
- name: alertmanager
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/observability/quadlet
units: [alertmanager]
# The Alertmanager config ships from Prometheus' directory, and the Signal
# relay is a HOST unit this handler installs: a change to either reaches
# the machine only through an alertmanager deploy.
files:
- shared/prometheus/alertmanager.yml
- scripts/signal-relay.py
- scripts/signal-relay.service
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
tests/docs_completeness.bats]
probe: 127.0.0.1:9093
# --- Identity -------------------------------------------------------------
- name: authentik-postgres
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/authentik/quadlet
units: [authentik-postgres]
deployed_by: authentik-server
tests: [tests/authentik.bats]
- name: authentik-server
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/authentik/quadlet
units: [authentik-server]
tests: [tests/authentik.bats]
dns: [auth.franksdank.cloud]
- name: authentik-worker
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/authentik/quadlet
units: [authentik-worker]
deployed_by: authentik-server
tests: [tests/authentik.bats]
- name: authentik-outpost
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/authentik/quadlet
units: [authentik-outpost]
tests: [tests/authentik.bats]
- name: authentik-outpost-ldap
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/authentik/quadlet
units: [authentik-outpost-ldap]
tests: [tests/authentik.bats]
# --- The auth chain's edge, then the edge itself --------------------------
- name: anubis
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/anubis/quadlet
units: [anubis]
dirs: [dankclaw/anubis]
tests: [tests/anubis.bats, tests/caddyfile.bats, tests/homepage_caddy.bats]
- name: caddy
hosts: [dankclaw]
kind: quadlet
unit_dir: dankclaw/caddy/quadlet
units: [caddy]
dirs: [dankclaw/caddy]
files: [dankclaw/Caddyfile]
# Retiring the old split proxy selects Caddy: it must remove the installed
# unit and container before it can claim 80/443.
retires: [dankclaw/gitea/quadlet/gitea-caddy.container]
tests: [tests/caddy_quadlet.bats, tests/caddyfile.bats,
tests/homepage_caddy.bats, tests/docs_completeness.bats]
dns:
- franksdank.cloud
- www.franksdank.cloud
- dankclaw.franksdank.cloud
- dankserver.franksdank.cloud
probe: 127.0.0.1:80
- name: homepage
hosts: [dankclaw]
kind: host
dirs: [shared/homepage]
files:
- scripts/homepage-status.py
- scripts/homepage-status.service
- scripts/homepage-status.timer
tests: [tests/homepage_caddy.bats, tests/homepage_config.bats,
tests/homepage_deploy.bats, tests/homepage_pages.bats,
tests/homepage_checker.bats]
# --- Dankserver media stack ----------------------------------------------
# The romm trio is one handler and one directory: every file in it selects
# all three, exactly as the deployer restarts all three.
- name: romm-mariadb
hosts: [dankserver]
kind: quadlet
unit_dir: dankserver/romm/quadlet
units: [romm-mariadb]
deployed_by: romm
dirs: [dankserver/romm]
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
probe: 192.168.0.235:3306
- name: romm-redis
hosts: [dankserver]
kind: quadlet
unit_dir: dankserver/romm/quadlet
units: [romm-redis]
deployed_by: romm
dirs: [dankserver/romm]
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
- name: romm
hosts: [dankserver]
kind: quadlet
unit_dir: dankserver/romm/quadlet
units: [romm]
dirs: [dankserver/romm]
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
dns: [romm.franksdank.cloud]
probe: 192.168.0.235:8085
- name: copyparty
hosts: [dankserver]
kind: quadlet
unit_dir: dankserver/copyparty/quadlet
units: [copyparty]
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
dns: [files.franksdank.cloud]
probe: 192.168.0.235:3923
- name: jellyfin
hosts: [dankserver]
kind: quadlet
unit_dir: dankserver/jellyfin/quadlet
units: [jellyfin]
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
dns: [jelly.franksdank.cloud]
probe: 192.168.0.235:8096
- name: calibre-web
hosts: [dankserver]
kind: quadlet
unit_dir: dankserver/calibre-web/quadlet
units: [calibre-web]
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
dns: [books.franksdank.cloud]
probe: 192.168.0.235:8083
- name: calibre-server
hosts: [dankserver]
kind: quadlet
unit_dir: dankserver/calibre-server/quadlet
units: [calibre-server]
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
- name: plex
hosts: [dankserver]
kind: quadlet
unit_dir: dankserver/plex/quadlet
units: [plex]
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
dns: [plex.franksdank.cloud, plex.franksdank.casa]
probe: 192.168.0.235:32400
- name: tautulli
hosts: [dankserver]
kind: quadlet
unit_dir: dankserver/tautulli/quadlet
units: [tautulli]
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
probe: 192.168.0.235:8181
# --- FrankBook-Pro: the *arr stack inside the podman machine --------------
# Every container rides gluetun's netns, and that coupling is DERIVED: each
# arr's unit file says `Network=gluetun.container`, so a gluetun unit change
# selects the whole stack with no declaration here.
- name: gluetun
hosts: [frankbook-pro]
kind: vm-quadlet
unit_dir: frankbook-pro/gluetun/quadlet
units: [gluetun]
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
- name: prowlarr
hosts: [frankbook-pro]
kind: vm-quadlet
unit_dir: frankbook-pro/prowlarr/quadlet
units: [prowlarr]
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
dns: [prowlarr.franksdank.cloud]
probe: 192.168.0.112:9696
- name: qbittorrent
hosts: [frankbook-pro]
kind: vm-quadlet
unit_dir: frankbook-pro/qbittorrent/quadlet
units: [qbittorrent]
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
dns: [qbittorrent.franksdank.cloud]
probe: 192.168.0.112:8080
- name: radarr
hosts: [frankbook-pro]
kind: vm-quadlet
unit_dir: frankbook-pro/radarr/quadlet
units: [radarr]
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
dns: [radarr.franksdank.cloud]
probe: 192.168.0.112:7878
- name: sonarr
hosts: [frankbook-pro]
kind: vm-quadlet
unit_dir: frankbook-pro/sonarr/quadlet
units: [sonarr]
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
probe: 192.168.0.112:8989
- name: lidarr
hosts: [frankbook-pro]
kind: vm-quadlet
unit_dir: frankbook-pro/lidarr/quadlet
units: [lidarr]
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
dns: [lidarr.franksdank.cloud]
probe: 192.168.0.112:8686
- name: readarr
hosts: [frankbook-pro]
kind: vm-quadlet
unit_dir: frankbook-pro/readarr/quadlet
units: [readarr]
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
dns: [readarr.franksdank.cloud]
probe: 192.168.0.112:8787
# --- The LLM fleet (last: never implied by a generic change) --------------
- name: frank-dragoon
hosts: [dankclaw]
kind: root-unit
files:
- scripts/frank-dragoon.env
- scripts/frank-dragoon.service
- scripts/serve-frank-dragoon.sh
health: http://127.0.0.1:7770/health
wait: 1200
tests: [tests/frank_dragoon_unit.bats, tests/llm_units.bats,
tests/llm_deploy_gates.bats, tests/llm_secrets.bats,
tests/scripts_syntax.bats, tests/docs_completeness.bats]
scrape:
- job: frank-dragoon
target: host.containers.internal:7770
metrics_path: /metrics
labels: {service: frank-dragoon}
probe: 127.0.0.1:7770
- name: frank-zealot
hosts: [dankclaw]
kind: root-unit
files:
- scripts/frank-zealot.env
- scripts/frank-zealot.service
- scripts/serve-frank-zealot.sh
health: http://127.0.0.1:7771/health
wait: 900
tests: [tests/frank_zealot_unit.bats, tests/llm_units.bats,
tests/llm_deploy_gates.bats, tests/llm_secrets.bats,
tests/scripts_syntax.bats]
scrape:
- job: frank-zealot
target: host.containers.internal:7771
metrics_path: /metrics
labels: {service: frank-zealot}
probe: 127.0.0.1:7771
- name: frank-archon
hosts: [dankserver]
kind: root-unit
files:
- scripts/frank-archon.env
- scripts/frank-archon.service
- scripts/serve-frank-archon.sh
# The vendored chat template the engine renders with.
- scripts/qwen38-froggeric-v22.5.jinja
health: http://127.0.0.1:7770/health
wait: 900
tests: [tests/frank_archon_unit.bats, tests/llm_units.bats,
tests/llm_deploy_gates.bats, tests/llm_secrets.bats,
tests/scripts_syntax.bats]
scrape:
- job: frank-archon
target: 192.168.0.235:7770
metrics_path: /metrics
labels: {service: frank-archon}
probe: 192.168.0.235:7770
+1 -1
View File
@@ -122,7 +122,7 @@ UNIT_DIR="$REPO_ROOT/dankclaw/authentik/quadlet"
@test "the homepage vhost is www, guarded by Anubis" {
local block
block=$(awk '/^www\.franksdank\.cloud[[:space:]]*\{/,/^}/' "$CADDYFILE")
echo "$block" | grep -q 'reverse_proxy anubis:8923'
echo "$block" | grep -q 'import guarded'
}
@test "the naked domain redirects to www at the TLS edge" {
+11
View File
@@ -251,3 +251,14 @@ CADDYFILE="$REPO_ROOT/dankclaw/Caddyfile"
grep -q -- '--server.http.listen-addr=0.0.0.0:12345' "$unit"
grep -q '^PublishPort=127.0.0.1:12345:12345$' "$unit"
}
@test "the guard chain is a named snippet the guarded vhosts import" {
# Rank 6: the Anubis pre-auth chain (reverse_proxy anubis:8923 with the
# X-Real-Ip header_up) existed as nine verbatim copies. One snippet,
# per-vhost imports; a chain edit is a one-line change.
grep -q '^(guarded) {' "$CADDYFILE"
local imports
imports=$(grep -c $'^\timport guarded$' "$CADDYFILE")
[ "$imports" -ge 9 ]
[ "$(grep -c 'reverse_proxy anubis:8923' "$CADDYFILE")" -eq 1 ]
}
+32 -4
View File
@@ -13,11 +13,15 @@ load helpers
[ "$status" -eq 0 ]
}
@test "changed-test wrapper maps a compose change to compose and environment tests" {
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths dankserver/compose.yaml
@test "changed-test wrapper maps a shared config change to its service seams" {
# The compose era is retired (rank 7): the former compose fixture here
# mapped dankserver/compose.yaml, a file that no longer exists. The
# equivalent contract now: a service config change maps to the
# service's seams and nothing unrelated.
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths shared/searxng/settings.yml
[ "$status" -eq 0 ]
[[ "$output" == *"tests/compose_validity.bats"* ]]
[[ "$output" == *"tests/env_consistency.bats"* ]]
[[ "$output" == *"tests/searxng_limiter.bats"* ]]
[[ "$output" == *"tests/searxng_quadlet.bats"* ]]
[[ "$output" != *"tests/homepage_pages.bats"* ]]
}
@@ -47,3 +51,27 @@ load helpers
[[ "$output" != *"tests/frank_medium_unit.bats"* ]]
[[ "$output" != *"tests/frank_archon_unit.bats"* ]]
}
@test "changed-test wrapper maps the frankbook unit to the frankbook and deploy-flow seams" {
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths frankbook-pro/sonarr/quadlet/sonarr.container
[ "$status" -eq 0 ]
[[ "$output" == *"tests/frankbook_quadlet.bats"* ]]
[[ "$output" == *"tests/deployment_flow.bats"* ]]
[[ "$output" == *"tests/deployment_scoping.bats"* ]]
[[ "$output" != *"tests/frank_archon_unit.bats"* ]]
}
@test "changed-test wrapper maps the inventory itself to its seam" {
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths shared/services.yml
[ "$status" -eq 0 ]
[[ "$output" == *"tests/service_inventory.bats"* ]]
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths scripts/service-map.py
[ "$status" -eq 0 ]
[[ "$output" == *"tests/service_inventory.bats"* ]]
}
@test "changed-test wrapper refuses an unmapped path" {
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths newdir/file.txt
[ "$status" -eq 2 ]
[[ "$output" == *"no focused-test mapping"* ]]
}
+129
View File
@@ -0,0 +1,129 @@
#!/usr/bin/env bash
# deploy_sandbox.bash — the deploy-handler seam's fixture builder.
#
# The seam (tests/llm_deploy_gates.bats) EXECUTES scripts/deploy-remote.sh's
# LLM handlers with every host touchpoint stubbed (tests/stubs/) and every
# artifact pointed at a fixture, so the assertions are the handler's exit
# status, its refusal text, and what it ASKED systemd/podman to do.
#
# Usage:
# source tests/deploy_sandbox.bash
# sandbox_setup
# run_handler dankserver frank-archon
# sandbox_teardown
#
# After sandbox_setup the caller owns:
# $STAGE the temp root (payload/, bin/, systemd/, state/)
# $REMOTE_DIR the sandbox checkout (a git clone; the handler
# cds into it, so relative paths resolve there)
# $STAGE/systemctl.log every systemctl invocation
# $STAGE/podman.log every podman invocation
# $STAGE/systemd/<unit>.service the unit link a green deploy leaves behind
# Env the caller sets per case: STUB_IMAGE ("" = empty storage),
# STUB_MAIN_PID (0 = a stray process owns the port), the LLM_* overrides.
sandbox_setup() {
STAGE="$(mktemp -d)"
export HOME="$STAGE/home"
mkdir -p "$HOME" "$STAGE/state" "$STAGE/systemd" "$STAGE/bin"
# Host touchpoints: the seam's stubs, on PATH under their real names.
local stub repo_root
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
for stub in "$repo_root"/tests/stubs/*; do
ln -s "$stub" "$STAGE/bin/$(basename "$stub")"
done
export PATH="$STAGE/bin:$PATH"
export STUB_SYSTEMCTL_LOG="$STAGE/systemctl.log"
export STUB_PODMAN_LOG="$STAGE/podman.log"
# A deploy marks state; a sandbox with no git identity would make the
# identity, not the handler, the thing under test.
export GIT_AUTHOR_NAME=seam GIT_AUTHOR_EMAIL=seam@localhost
export GIT_COMMITTER_NAME=seam GIT_COMMITTER_EMAIL=seam@localhost
export SYSTEMD_SYSTEM_DIR="$STAGE/systemd"
export DEPLOY_STATE_DIR="$STAGE/state"
# run_privileged pipes the password from FRANK_SUDO_PW into `sudo -S --`;
# the stub drops it and execs, so no privilege gate hides a handler bug.
export FRANK_SUDO_PW=test
# Fixture payload: 3 MiB where a size floor applies, 10 bytes elsewhere.
mkdir -p "$STAGE/models/tokenizer"
truncate -s 3145728 "$STAGE/models/payload.rad"
truncate -s 10 "$STAGE/models/ck.hgn"
truncate -s 10 "$STAGE/models/ngram.hgn"
truncate -s 10 "$STAGE/models/vision.hgn"
printf 'x\n' > "$STAGE/models/llama-server"
chmod +x "$STAGE/models/llama-server"
truncate -s 10 "$STAGE/models/model.gguf"
truncate -s 10 "$STAGE/models/mmproj.gguf"
local tf
for tf in tokenizer.json chat_template.jinja tokenizer_config.json \
merges.txt vocab.json generation_config.json; do
printf 'x\n' > "$STAGE/models/tokenizer/$tf"
done
# $HOME/serverAppConfigs is the checkout the handler works in: a git
# WORKTREE of the caller's checkout, so the handler runs the EXACT files
# under test, and it is a real git repo: the deployer refuses a revision
# it cannot resolve. The handler cds into REMOTE_DIR, so every relative
# path it uses resolves there.
#
# HEAD is committed state, so the seam asserts on what is in history —
# the same rule as the whole suite, and the reason an uncommitted
# deployer change is invisible here until it is committed.
git -C "$repo_root" worktree add -q --detach "$HOME/serverAppConfigs" HEAD
export REMOTE_DIR="$HOME/serverAppConfigs"
# Every endpoint variable is env-overridable, so the handlers see the
# fixtures and never the real /home/dank paths.
export STUB_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:deadbeef
export LLM_DRAGOON_IMAGE="$STUB_IMAGE"
export LLM_DRAGOON_CKPT="$STAGE/models/ck.hgn" LLM_DRAGOON_CKPT_BYTES=10
export LLM_DRAGOON_NGRAM="$STAGE/models/ngram.hgn" LLM_DRAGOON_NGRAM_BYTES=10
export LLM_DRAGOON_VISION="$STAGE/models/vision.hgn" LLM_DRAGOON_VISION_BYTES=10
export LLM_DRAGOON_TOK_DIR="$STAGE/models/tokenizer"
export LLM_ZEALOT_RUNTIME="$STAGE/models/llama-server"
export LLM_ZEALOT_MODEL_FILE="$STAGE/models/model.gguf"
export LLM_ZEALOT_MMPROJ="$STAGE/models/mmproj.gguf"
export LLM_ARCHON_ENV_FILE="$REMOTE_DIR/scripts/frank-archon.env"
# The archon handler reads the served image out of the unit's own env
# file, so the stub's storage must hold exactly that ref.
ARCHON_IMAGE="$(sed -nE 's/^FRANK_ARCHON_IMAGE=(.+)$/\1/p' "$LLM_ARCHON_ENV_FILE")"
}
sandbox_teardown() {
[[ -z "${STAGE:-}" ]] && return 0
# A worktree leaves a registration in the caller's .git/worktrees, so
# remove it through git, not just the directory.
[[ -d "$STAGE/home/serverAppConfigs" ]] &&
git -C "$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" \
worktree remove --force "$STAGE/home/serverAppConfigs"
rm -rf "$STAGE"
}
run_handler() {
local target="$1" service="$2"
run bash "$REMOTE_DIR/scripts/deploy-remote.sh" \
"$target" "$(git -C "$REMOTE_DIR" rev-parse HEAD)" "$service"
}
# archon_payload_env — the unit's env file with the payload coordinates
# redirected into the sandbox (and the size floor optionally lowered, which
# is how a test builds the truncated-.rad case).
archon_payload_env() {
local re="s#^FRANK_ARCHON_MODEL_DIR=.*#FRANK_ARCHON_MODEL_DIR=$STAGE/models#;"
re+=" s#^FRANK_ARCHON_MODEL_FILE=.*#FRANK_ARCHON_MODEL_FILE=payload.rad#"
if [[ -n "${1:-}" ]]; then
re+="; s#^FRANK_ARCHON_RAD_MIN_MIB=.*#FRANK_ARCHON_RAD_MIN_MIB=$1#"
fi
sed -E "$re" "$LLM_ARCHON_ENV_FILE" > "$STAGE/archon.env"
export LLM_ARCHON_ENV_FILE="$STAGE/archon.env"
}
restarts_requested() {
# grep -c prints 0 and exits 1 on no match; the count, not the status,
# is what the assertions read.
local n=0
[[ -f "$STAGE/systemctl.log" ]] && n="$(grep -c 'restart ' "$STAGE/systemctl.log")"
printf '%s\n' "$n"
}
+4 -1
View File
@@ -78,7 +78,10 @@ load helpers
}
@test "remote runner health-checks an inference endpoint" {
grep -q '127.0.0.1:7770/health' "$REPO_ROOT/scripts/deploy-remote.sh"
# start_root_unit carries the health gate for all three LLM units; the
# contract is that each endpoint's start names its port, and 7770 is
# frank-dragoon's.
grep -q 'start_root_unit frank-dragoon 7770' "$REPO_ROOT/scripts/deploy-remote.sh"
}
@test "deploy.sh pulls the merged checkout on the target" {
+3 -1
View File
@@ -1,5 +1,7 @@
#!/usr/bin/env bats
# Seam: documentation completeness.
# (compose era retired 2026-10-13; the sweep below now skips trash so a
# revived compose file stays covered without resurrecting retired services)
# Every service defined in a compose.yaml must have a matching documentation
# page in serverAppConfigs-docs/ and must be reachable from the index. This is the "docs first"
# contract of this repo: no service ships without documentation.
@@ -18,7 +20,7 @@ load helpers
return 1
fi
done < <(services_in "$f")
done < <(find "$REPO_ROOT" -name 'compose.yaml' -not -path '*/.git/*' | sort)
done < <(find "$REPO_ROOT" -name 'compose.yaml' -not -path '*/.git/*' -not -path '*/.AGENTTRASH/*' | sort)
}
@test "docs index links every docs page" {
+3 -24
View File
@@ -1,33 +1,12 @@
#!/usr/bin/env bats
# Seam: environment variable consistency and secret hygiene.
# Every ${VAR} referenced in a compose.yaml or Caddyfile must be defined in the
# matching .env.example. .env.example files must contain only placeholders, and
# Every ${VAR} referenced in a Caddyfile must be defined in the matching
# .env.example. .env.example files must contain only placeholders, and
# the live gitea database password must never appear in tracked files.
# (The compose.yaml sweeps retired with the last compose file on 2026-10-13.)
load helpers
@test "every compose.yaml has a sibling .env.example" {
while IFS= read -r f; do
local envfile
envfile="$(dirname "$f")/.env.example"
[ -f "$envfile" ] || { echo "missing $envfile for $f"; return 1; }
done < <(find "$REPO_ROOT" -name 'compose.yaml' -not -path '*/.git/*' | sort)
}
@test "every \${VAR} in compose.yaml is defined in the sibling .env.example" {
while IFS= read -r f; do
local envfile
envfile="$(dirname "$f")/.env.example"
while IFS= read -r var; do
[ -z "$var" ] && continue
if ! env_vars_defined "$envfile" | grep -qx "$var"; then
echo "$f references \${$var} but $envfile does not define it"
return 1
fi
done < <(env_vars_in "$f")
done < <(find "$REPO_ROOT" -name 'compose.yaml' -not -path '*/.git/*' | sort)
}
@test ".env.example files contain only placeholder values" {
while IFS= read -r f; do
has_placeholder_secret "$f"
File diff suppressed because it is too large Load Diff
+143 -61
View File
@@ -13,13 +13,17 @@
# the serve script ENFORCES it, the served window is what models.yml must
# quote, the host-networking contract (no --publish), the image gate on a
# name@sha256 ref, the .rad payload gate with its size floor, the
# chat-template override mount, the self-healing law (the run returns before
# the engine is up, so the serve script must wait on the container for
# Restart=always to mean anything), the deployer's false-green MainPID check,
# the vendored hfget fetch path (no token for this ungated repo), and the
# chat-template override mount, the self-healing contract (`podman run`
# returns before the engine is up, so the serve script must stay attached
# for Restart=always to mean anything), the deployer's false-green MainPID
# check, the vendored hfget fetch path (no token for this ungated repo), and the
# client contract (reasoning auto -> reasoning_content; models.yml maps the
# field on the harness side, not here).
# The serve script is EXECUTED against fixtures (stub podman, stub device
# nodes, stub ttm sysfs knobs, a temp payload dir) so the gates are observed
# as the operator observes them: exit status and the words on stderr. The
# knobs that make that possible (FRANK_ARCHON_DEV_KFD, _DEV_DRI,
# _TTM_SYSFS_DIR, _LOG) default to the production paths.
load helpers
UNIT="$REPO_ROOT/scripts/frank-archon.service"
@@ -30,6 +34,133 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-archon-swift15.sh"
BUILD="$REPO_ROOT/scripts/convert-swift15-w4dn8.sh"
HFGET="$REPO_ROOT/scripts/hfget.sh"
TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
#
# Fixtures: a full env derived from the SHIPPED frank-archon.env with only
# the host paths redirected, so a payload rename moves the env, not this file.
setup() {
STAGE="$(mktemp -d)"
MODELS="$STAGE/models"
STATE="$STAGE/state"
TTM="$STAGE/ttm"
mkdir -p "$MODELS" "$STATE" "$TTM" "$STAGE/bin" "$STAGE/dri"
touch "$STAGE/kfd" "$STAGE/template.jinja"
# A 3 MiB .rad clears the 2 MiB floor the fixture env pins.
dd if=/dev/zero of="$MODELS/payload.rad" bs=1048576 count=3 status=none
# 200 GiB of pages, the shipped cap's value, in 4 KiB pages.
printf '52428800\n' > "$TTM/pages_limit"
cat > "$STAGE/bin/podman" <<'STUB'
#!/usr/bin/env bash
case "$1" in
image)
case " ${STUB_IMAGE:-} " in
*" $3 "*) exit 0 ;;
*) exit 1 ;;
esac ;;
rm) exit 0 ;;
run) echo "PODMAN $*" >> "${STUB_ARGV:-/dev/null}"; exit 0 ;;
*) exit 0 ;;
esac
STUB
chmod +x "$STAGE/bin/podman"
IMAGE="$(sed -nE 's/^FRANK_ARCHON_IMAGE=(.+)$/\1/p' "$ENV_FILE")"
# The fixture env IS the shipped env with only the host paths and the
# size floor redirected, so a payload rename moves frank-archon.env and
# this file follows with it.
local re="s#^FRANK_ARCHON_MODEL_DIR=.*#FRANK_ARCHON_MODEL_DIR=$MODELS#;"
re+=" s#^FRANK_ARCHON_MODEL_FILE=.*#FRANK_ARCHON_MODEL_FILE=payload.rad#;"
re+=" s#^FRANK_ARCHON_RAD_MIN_MIB=.*#FRANK_ARCHON_RAD_MIN_MIB=2#;"
re+=" s#^FRANK_ARCHON_TEMPLATE=.*#FRANK_ARCHON_TEMPLATE=$STAGE/template.jinja#;"
re+=" s#^FRANK_ARCHON_STATE_DIR=.*#FRANK_ARCHON_STATE_DIR=$STATE#;"
re+=" s#^FRANK_ARCHON_TTM_MIN_MIB=.*#FRANK_ARCHON_TTM_MIN_MIB=150000#"
sed -E "$re" "$ENV_FILE" > "$STAGE/serve.env"
{
echo "FRANK_ARCHON_TTM_SYSFS=$TTM"
echo "FRANK_ARCHON_DEV_KFD=$STAGE/kfd"
echo "FRANK_ARCHON_DEV_DRI=$STAGE/dri"
echo "FRANK_ARCHON_LOG=$STAGE/serve.log"
echo "STUB_IMAGE=$IMAGE"
echo "STUB_ARGV=$STAGE/argv.txt"
} >> "$STAGE/serve.env"
}
teardown() {
rm -rf "$STAGE"
}
# serve_run — execute the serve script against the fixtures.
serve_run() {
run env PATH="$STAGE/bin:$PATH" FRANK_ARCHON_ENV_FILE="$STAGE/serve.env" \
bash "$SERVE"
}
@test "the serve script refuses when the image is not in storage" {
# The env file wins over the ambient environment (the serve script
# sources it with `set -a`), so the stub's storage is emptied in the file.
sed -i 's#^STUB_IMAGE=.*#STUB_IMAGE=#' "$STAGE/serve.env"
serve_run
[ "$status" -ne 0 ]
[[ "$output" == *"not in podman storage"* ]]
[[ "$output" == *"podman pull"* ]]
}
@test "the serve script refuses a missing .rad and names its builder" {
rm -f "$MODELS/payload.rad"
serve_run
[ "$status" -ne 0 ]
[[ "$output" == *"container: missing"*"$MODELS/payload.rad"* ]]
[[ "$output" == *"scripts/convert-swift15-w4dn8.sh"* ]]
}
@test "the serve script refuses a .rad below the size floor, naming both sizes" {
# A truncated rad-convert is a guaranteed long-timeout boot; the floor is
# the fast failure (FRANK_ARCHON_RAD_MIN_MIB).
dd if=/dev/zero of="$MODELS/payload.rad" bs=1048576 count=1 status=none
serve_run
[ "$status" -ne 0 ]
[[ "$output" == *"1 MiB"* ]]
[[ "$output" == *"2 MiB"* ]]
}
@test "the serve script refuses an absent chat template" {
# The engine is started with --override-chat-template; a missing file is
# a container that dies at argv parse, not a fast refusal.
rm -f "$STAGE/template.jinja"
serve_run
[ "$status" -ne 0 ]
[[ "$output" == *"template: missing"*"$STAGE/template.jinja"* ]]
}
@test "the serve script refuses when the ttm pages cap is below the pinned ledger" {
printf '1048576\n' > "$TTM/pages_limit" # 4 GiB, under the ~147 GiB ledger
serve_run
[ "$status" -ne 0 ]
[[ "$output" == *"ttm pages cap is 4096 MiB"* ]]
[[ "$output" == *"150000"* ]]
}
@test "the serve script refuses when the GPU device nodes are absent" {
rm -f "$STAGE/kfd"
serve_run
[ "$status" -ne 0 ]
[[ "$output" == *"GPU device nodes absent"*"$STAGE/kfd"* ]]
}
@test "the serve script runs the engine with the .rad, the template and TP=2" {
serve_run
[ "$status" -eq 0 ]
argv="$(cat "$STAGE/argv.txt")"
grep -q -- "--model /models/payload.rad" <<<"$argv"
grep -q -- "-v $MODELS:/models:ro" <<<"$argv"
grep -q -- "--override-chat-template /templates/template.jinja" <<<"$argv"
grep -q -- "--tp $(sed -nE 's/^FRANK_ARCHON_TP=([0-9]+)$/\1/p' "$ENV_FILE")" <<<"$argv"
grep -q -- "--tp-wire $(sed -nE 's/^FRANK_ARCHON_TP_WIRE=(.+)$/\1/p' "$ENV_FILE")" <<<"$argv"
grep -q -- "--kv-cache-dtype $(sed -nE 's/^FRANK_ARCHON_KV_DTYPE=(.+)$/\1/p' "$ENV_FILE")" <<<"$argv"
# The container name is the unit's ExecStop target, and the host-network
# contract publishes no port.
grep -q -- "--name $(sed -nE 's/^FRANK_ARCHON_CONTAINER=(.+)$/\1/p' "$ENV_FILE")" <<<"$argv"
grep -q -- "--network host" <<<"$argv"
! grep -q -- "--publish" <<<"$argv"
}
@test "frank-archon unit, serve script, env, fetch and build scripts and template exist" {
[ -f "$UNIT" ]
@@ -122,11 +253,11 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
[ "$ml" -ge 177356 ]
# Concurrency: 2 slots x the served window is the capacity contract, and
# the fused TP all-reduce is declared over seqs x (1 + spec) = 6 rows,
# under the 25-row bound at which the wht6 wire stays bit-exact. The
# "seqs 2 cannot boot" law belongs to the retired tcclaviger vLLM fork
# (torch.compile query_start_loc constraint, 2026-10-02); radiance booted
# seqs 2 in the fp8-native era and the 2026-10-06 sweep had seqs 2 and 3
# tied inside the +-3% noise floor.
# under the 25-row bound at which the wht6 wire stays bit-exact. A
# "this slot count cannot boot" refusal is per-runtime (compiler shape
# specialization rejects counts the engine running here accepts), so it
# must be re-tested, not inherited; measurements here put seqs 2 and 3
# inside each other's noise floor.
grep -q '^FRANK_ARCHON_MAX_SEQS=2$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE"
@@ -134,8 +265,8 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
# expert_tiered placement: vram_weights + the pinned host pool serve the
# per-rank resident set TOGETHER, so cutting the VRAM slab to pay for the
# KV pool moves the pool with it. KV is UNQUANTIZED bf16 (the engine takes
# only bf16|fp8 — no per-tensor K/V split), the PLE n-gram table is pinned
# in host RAM per Frank's law.
# only bf16|fp8 — no per-tensor K/V split), the PLE n-gram table rides
# pinned host RAM per the placement knob.
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=44928$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE"
@@ -255,22 +386,6 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
! grep -q 'FRANK_ARCHON_TOOL_PARSER' "$ENV_FILE"
}
@test "the serve script gates the image, the .rad payload, the template and the GPU nodes" {
# The image gate is the digest ref itself; the payload gate checks the
# .rad file and its size floor — a truncated convert would otherwise be
# a guaranteed long-timeout boot instead of a fast failure. The ttm
# preflight refuses to start under a pages cap below the pinned ledger.
grep -q 'FRANK_ARCHON_IMAGE' "$SERVE"
grep -q 'podman image exists' "$SERVE"
grep -q 'FRANK_ARCHON_MODEL_FILE' "$SERVE"
grep -q 'FRANK_ARCHON_RAD_MIN_MIB' "$SERVE"
grep -q 'FRANK_ARCHON_TEMPLATE' "$SERVE"
grep -q 'FRANK_ARCHON_TTM_MIN_MIB' "$SERVE"
grep -q 'pages_limit' "$SERVE"
grep -q 'pages_max' "$SERVE"
grep -q '/dev/kfd' "$SERVE"
grep -q -- '--device /dev/dri' "$SERVE"
}
@test "the serve script clears stale containers and self-heals via podman wait" {
# A crashed unit can leave the container name registered forever after;
@@ -304,36 +419,3 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
! grep -qE '^(TEMP|HOST|GPU_MAX_HEAP_SIZE)=' "$ENV_FILE"
! grep -qE '^(export )?(TEMP|HOST|GPU_MAX_HEAP_SIZE)=' "$SERVE"
}
@test "the deployer gates the image, payload and containers, then health-checks" {
# The deployer refuses when the image is absent from dank's rootless
# storage or the .rad is missing/too small — then clears stale
# containers (including the legacy identities), waits out the port,
# relinks/enables/restarts the root unit and health-gates it.
grep -q 'LLM_ARCHON_CONTAINER=frank-archon$' "$REMOTE_DEPLOY"
grep -q 'FRANK_ARCHON_IMAGE' "$REMOTE_DEPLOY"
grep -q 'podman image exists' "$REMOTE_DEPLOY"
grep -q 'FRANK_ARCHON_MODEL_FILE' "$REMOTE_DEPLOY"
grep -q 'FRANK_ARCHON_RAD_MIN_MIB' "$REMOTE_DEPLOY"
grep -q 'frank-archon-qwen' "$REMOTE_DEPLOY"
grep -q 'qwen-fp8' "$REMOTE_DEPLOY"
grep -q 'glm-probe' "$REMOTE_DEPLOY"
grep -q 'wait_port_free 7770' "$REMOTE_DEPLOY"
grep -q 'relink_unit frank-archon' "$REMOTE_DEPLOY"
grep -q 'systemctl enable frank-archon.service' "$REMOTE_DEPLOY"
grep -q 'systemctl restart frank-archon.service' "$REMOTE_DEPLOY"
# The health gate is a bare curl: the HTTP listener binds AFTER the
# full load (main.cpp — engine.start() then engine.serve()), so during
# load the port refuses connections and a bare curl is honest. The
# FreeToken-era body read ('status ok') is gone with that engine.
grep -q 'curl -fsS http://127.0.0.1:7770/health' "$REMOTE_DEPLOY"
! grep -q '"status":"ok"' "$REMOTE_DEPLOY"
}
@test "the deployer will not report green on a stray port-holder" {
# The 2026-09-26 false green: a leftover probe held :7770 and satisfied
# the health curl while the real unit was stopped. The deploy is only
# green when the unit's own MainPID is live behind the health answer.
grep -q 'systemctl show -p MainPID --value frank-archon.service' "$REMOTE_DEPLOY"
grep -q 'stray process holds the port' "$REMOTE_DEPLOY"
}
+28 -24
View File
@@ -10,11 +10,11 @@
# exact strings the engine reads: the image DIGEST, the checkpoint/n-gram/
# vision file names AND byte counts inside the /models mount, and the
# tokenizer's flat-file shape (a HF cache snapshot's symlinks dangle inside
# the container). A serve gate and a deployer gate that disagree is the
# 2026-10-01 crash-loop; both sides are pinned here, plus the one engine
# law the card is emphatic about: HALOGEN_SPEC_ADAPT=0 — the default
# adaptive policy silently disables the MTP head per request and HALVES
# decode.
# the container). A serve gate and a deployer gate that disagree let a
# deploy pass what the unit would refuse, so both sides are pinned here —
# plus the one engine behavior the card is emphatic about:
# HALOGEN_SPEC_ADAPT=0 — the default adaptive policy silently disables the
# MTP head per request and HALVES decode.
#
# Everything is tested from what an operator or supervisor can observe —
# the command line podman would run, the exit code, the deployer's error
@@ -45,6 +45,13 @@ podman_stub() {
#!/usr/bin/env bash
case "$1" in
images) printf '%s\n' "${STUB_DIGESTS:-}" ;;
image)
# `podman image exists REF` — exit 1 when storage lacks the ref,
# which is exactly the primitive the shared gate library calls.
case " ${STUB_DIGESTS:-} " in
*" $3 "*) exit 0 ;;
*) exit 1 ;;
esac ;;
rm) exit 0 ;;
run|wait) echo "PODMAN $*"; exit "${STUB_WAIT_RC:-0}" ;;
*) exit 0 ;;
@@ -110,10 +117,11 @@ good_env() {
@test "the head stays on: adaptive spec must stay 0 and the window shape pinned" {
# The card is emphatic: the default adaptive policy turns the MTP head
# off for whole requests after low acceptance and HALVES decode. The
# two-slot law does NOT carry over from the 27B engine — on this line
# batching and speculation coexist (the card's quickstart ships 2
# slots WITH the head), so 2 is concurrent capacity, not a tax.
# off for whole requests after low acceptance and HALVES decode. A
# "slots > 1 disables drafting" restriction belongs to the engine
# generation that imposed it — on this line batching and speculation
# coexist (the card's quickstart ships 2 slots WITH the head), so 2 is
# concurrent capacity, not a tax.
grep -qE '^HALOGEN_SPEC_ADAPT=0$' "$ENV_FILE"
grep -qE '^HALOGEN_KV_SLOTS=2$' "$ENV_FILE"
# The cap/timeout pair is coupled (FLAGS.md table): a request at the
@@ -179,7 +187,7 @@ good_env() {
@test "the serve script refuses a checkpoint of the wrong byte count" {
# A truncated curl exit-0 artifact must not boot; the Hub's byte
# count is the contract (fleet fetch law).
# count is the contract, any other size is a broken artifact.
podman_stub
good_env
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
@@ -197,7 +205,7 @@ good_env() {
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
FRANK_DRAGOON_NGRAM_TABLE=absent.hgn "$SERVE"
[ "$status" -ne 0 ]
[[ "$output" == *"n-gram-table"*"missing"* ]]
[[ "$output" == *"frank-dragoon: n-gram: missing at $TEST_TMP/absent.hgn"* ]]
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
FRANK_DRAGOON_NGRAM_BYTES=42 "$SERVE"
[ "$status" -ne 0 ]
@@ -213,7 +221,7 @@ good_env() {
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
FRANK_DRAGOON_VISION_TOWER=absent.hgn "$SERVE"
[ "$status" -ne 0 ]
[[ "$output" == *"vision-tower"*"missing"* ]]
[[ "$output" == *"frank-dragoon: vision: missing at $TEST_TMP/absent.hgn"* ]]
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
FRANK_DRAGOON_VISION_BYTES=42 "$SERVE"
[ "$status" -ne 0 ]
@@ -280,16 +288,16 @@ $(cat "$TEST_TMP/serve.log")"
! grep -q 'PODMAN wait' "$TEST_TMP/serve.log"
}
@test "the shipped env keeps the harness fit law inside the ceiling" {
@test "the shipped env keeps the harness fit margin inside the ceiling" {
# 262144 (the checkpoint's native window) is the per-slot hard ceiling
# (a longer prompt is a 400 naming the limit, never a silent
# truncation). The harness clamps max_tokens = min(45056, 217088 − est
# − 4096), compacts at 85 % of its declared 217088 window, and its
# chars/4 est UNDER-counts the rendered prompt by ~3 % on code/agent
# traffic (measured 4,450 tokens at est 152,938 on 2026-10-07 — with
# window == ceiling, every request over ~146k est was a guaranteed
# 400 before compaction could fire at 85 %). The 45k gap between the
# harness window and the engine ceiling must absorb that error at
# traffic — so a harness window equal to the engine ceiling would make
# every long request a guaranteed 400 before compaction could fire.
# The 45k gap between the harness window and the engine ceiling must
# absorb that error at
# both monotone maxima of est + budget (est ≤ 217088 − 4096):
# full budget at the compaction threshold: 1.03*184524 + 45056
# clamped budget at the clamp's zero point: 1.03*212992 + 0
@@ -328,11 +336,6 @@ $(cat "$TEST_TMP/serve.log")"
grep -qF "LLM_DRAGOON_CKPT_BYTES=\"\${LLM_DRAGOON_CKPT_BYTES:-$bytes}\"" "$DEPLOYER"
}
@test "the deployer points a missing artifact at the flashnext fetch script" {
grep -qE 'download-frank-dragoon-flashnext\.sh' "$DEPLOYER"
grep -qE 'podman pull' "$DEPLOYER"
}
@test "the stray sweep cannot take frank-zealot or the deployer down" {
# frank-zealot also listens on this host (7771, llama-server) and the
# deployer's own command line contains "frank-dragoon"; only the FULL
@@ -344,8 +347,9 @@ $(cat "$TEST_TMP/serve.log")"
}
@test "the deployed unit keeps the fleet's restart and boot-persistence shape" {
grep -qE '^Restart=(always|on-failure)$' "$UNIT"
grep -qE '^WantedBy=' "$UNIT"
# Restart=always and the WantedBy value are the FLEET parity seam
# (tests/llm_units.bats); what is endpoint-specific is which script the
# unit runs, under whom, and what a stop must reap.
grep -qE '^ExecStart=.*/serve-frank-dragoon\.sh$' "$UNIT"
grep -qE '^User=dank$' "$UNIT"
grep -qE '^LimitMEMLOCK=infinity$' "$UNIT"
+13 -17
View File
@@ -85,10 +85,12 @@ run_stage() {
[ -f "$ENV_FILE" ]
}
@test "frank-zealot is boot-persistent, self-healing and unprivileged" {
grep -q '^WantedBy=multi-user.target$' "$UNIT"
grep -q '^Restart=' "$UNIT"
grep -q '^RestartSec=' "$UNIT"
@test "frank-zealot runs unprivileged with the memlock budget and its env file" {
# Restart policy, RestartSec and WantedBy are the FLEET parity seam
# (tests/llm_units.bats). What is endpoint-specific: CUDA pins its
# staging buffers, so the memlock budget is load-bearing even though the
# weights are copied into VRAM, and the unit must name the serve script
# and the env file that sizes it.
grep -q '^User=dank$' "$UNIT"
grep -q '^LimitMEMLOCK=infinity$' "$UNIT"
grep -q '^ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-zealot.sh$' "$UNIT"
@@ -253,26 +255,20 @@ run_stage() {
}
@test "the serve script refuses a missing runtime, GGUF or projector" {
# The refusal is the shared gate library's: it names the artifact class,
# the exact path, and the script that produces it.
rm -f "$MODEL_DIR/$MM_STUB"
serve_argv
[ "$status" -ne 0 ]
[[ "$output" == *"frank-zealot: missing"* ]]
[[ "$output" == *"frank-zealot:"* ]]
[[ "$output" == *"mmproj"*"missing"*"$MM_STUB"* ]]
[[ "$output" == *"download-frank-zealot.sh"* ]]
rm -f "$RUNTIME_DIR/bin/llama-server"
serve_argv
[ "$status" -ne 0 ]
[[ "$output" == *"frank-zealot: missing"* ]]
}
@test "the deploy handler gates on binary, GGUF and projector, then MainPID-hardens health" {
grep -q 'deploy_llm_zealot' "$REMOTE_DEPLOY"
grep -q 'LLM_ZEALOT_RUNTIME' "$REMOTE_DEPLOY"
grep -q 'LLM_ZEALOT_MODEL_FILE' "$REMOTE_DEPLOY"
grep -q 'LLM_ZEALOT_MMPROJ' "$REMOTE_DEPLOY"
grep -q 'relink_unit frank-zealot' "$REMOTE_DEPLOY"
grep -q 'systemctl enable frank-zealot.service' "$REMOTE_DEPLOY"
grep -q '127.0.0.1:7771/health' "$REMOTE_DEPLOY"
grep -q 'frank-zealot.service has no MainPID' "$REMOTE_DEPLOY"
[[ "$output" == *"runtime"*"missing"*"llama-server"* ]]
[[ "$output" == *"build-llama-master-cuda.sh"* ]]
}
@test "the two endpoints keep separate ports and the large handler never pkills llama-server" {
+6
View File
@@ -4,6 +4,12 @@
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
# skip_if_no_podman — skip the test when podman is absent (the image gate
# asks real storage a real question)
skip_if_no_podman() {
command -v podman >/dev/null 2>&1 || skip "podman is not installed"
}
# yaml_parse_ok <file> — assert a compose file parses as valid YAML
yaml_parse_ok() {
python3 - "$1" <<'PYEOF'
+7 -2
View File
@@ -101,7 +101,10 @@ webui_service_yaml() {
}
@test "remote deployer accepts and dispatches hermes-webui" {
grep -q 'dankclaw:hermes-webui' "$REMOTE_DEPLOY"
# Acceptance is the inventory (rank 3): the old literal allow-list in
# the remote runner is retired.
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankclaw --service hermes-webui
[ "$status" -eq 0 ]
# Dispatched to the unit handler, never to compose: deploy_hermes installs
# the quadlets and retires compose-era containers.
grep -q 'hermes|hermes-dashboard|hermes-webui) deploy_hermes' "$REMOTE_DEPLOY"
@@ -111,5 +114,7 @@ webui_service_yaml() {
}
@test "top-level deployer accepts hermes-webui as a Dankclaw service" {
grep -q 'dankclaw:hermes-webui' "$DEPLOY"
# deploy.sh validates --service through the inventory's allows.
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankclaw --service hermes-webui
[ "$status" -eq 0 ]
}
+1 -1
View File
@@ -51,7 +51,7 @@ setup() {
# domain never enters the chain (it redirects at the TLS edge first).
local block
block=$(awk '/^www\.franksdank\.cloud[[:space:]]*\{/,/^}/' "$CADDY")
echo "$block" | grep -q 'reverse_proxy anubis:8923'
echo "$block" | grep -q 'import guarded'
# forwards accepted traffic to; Host survives the hop, so per-vhost
# routing resumes there — guarded vhosts proxy the whole request
# through Authentik, which forwards authenticated traffic back on the
+136
View File
@@ -0,0 +1,136 @@
#!/usr/bin/env bats
# Seam: the deployer's LLM handlers are the twin of the serve scripts'
# artifact gates (the 2026-10-01 crash-loop: a green deploy against a
# missing payload left a restart-looping unit and a dark endpoint).
#
# These tests EXECUTE scripts/deploy-remote.sh's LLM handlers in a sandbox
# (tests/deploy_sandbox.bash): every host touchpoint is a stub, the artifacts
# are fixtures, and the assertions are exit status, the refusal on stderr,
# and what the handler asked systemd to do. A handler that started the unit
# before checking its artifacts would pass a text-grep seam and fail here.
load helpers
load deploy_sandbox
setup() { sandbox_setup; }
teardown() { sandbox_teardown; }
@test "the dragoon handler refuses when the engine image is not in storage" {
export STUB_IMAGE=
run_handler dankclaw frank-dragoon
[ "$status" -ne 0 ]
[[ "$output" == *"frank-dragoon: image:"*"not in podman storage"* ]]
[ "$(restarts_requested)" = "0" ]
}
@test "the dragoon handler refuses a truncated checkpoint, pointing at the fetch" {
truncate -s 9 "$STAGE/models/ck.hgn"
run_handler dankclaw frank-dragoon
[ "$status" -ne 0 ]
[[ "$output" == *"checkpoint: is 9 bytes, expected 10"* ]]
# The refusal is only useful if it names the remedy: the fetch script
# re-verifies byte counts and hashes.
[[ "$output" == *"download-frank-dragoon-flashnext.sh"* ]]
[ "$(restarts_requested)" = "0" ]
}
@test "the dragoon handler refuses a symlinked tokenizer file" {
# Snapshot layouts link their members into place, and the link dangles
# inside the container mount — only regular files are servable.
rm "$STAGE/models/tokenizer/vocab.json"
ln -s /nowhere/vocab.json "$STAGE/models/tokenizer/vocab.json"
run_handler dankclaw frank-dragoon
[ "$status" -ne 0 ]
[[ "$output" == *"frank-dragoon: tokenizer: $STAGE/models/tokenizer/vocab.json is missing or a symlink"* ]]
[ "$(restarts_requested)" = "0" ]
}
@test "the dragoon handler starts the unit once every artifact gate passes" {
run_handler dankclaw frank-dragoon
[ "$status" -eq 0 ]
grep -q 'restart frank-dragoon.service' "$STAGE/systemctl.log"
grep -qx 'rm -f frank-dragoon frank-dragoon-halogen' "$STAGE/podman.log"
}
@test "the zealot handler refuses when the llama.cpp runtime is not executable" {
# The runtime is an operator-built binary; a tree that compiled but lost
# its exec bit must refuse exactly like a missing one.
rm "$STAGE/models/llama-server"
run_handler dankclaw frank-zealot
[ "$status" -ne 0 ]
[[ "$output" == *"frank-zealot: runtime: missing at $STAGE/models/llama-server"* ]]
[[ "$output" == *"cmake"* ]]
[ "$(restarts_requested)" = "0" ]
}
@test "the zealot handler refuses a missing projector — a text-only endpoint is not deployable" {
# The projector IS the vision capability: deploying without it silently
# ships a text-only endpoint.
rm "$STAGE/models/mmproj.gguf"
run_handler dankclaw frank-zealot
[ "$status" -ne 0 ]
[[ "$output" == *"frank-zealot: mmproj: missing at $STAGE/models/mmproj.gguf"* ]]
[ "$(restarts_requested)" = "0" ]
}
@test "the zealot handler starts the unit once every artifact gate passes" {
run_handler dankclaw frank-zealot
[ "$status" -eq 0 ]
grep -q 'restart frank-zealot.service' "$STAGE/systemctl.log"
}
@test "the archon handler refuses when the radiance image is not in storage" {
export STUB_IMAGE=
run_handler dankserver frank-archon
[ "$status" -ne 0 ]
[[ "$output" == *"frank-archon: image:"*"not in podman storage"* ]]
[[ "$output" == *"podman pull"* ]]
[ "$(restarts_requested)" = "0" ]
}
@test "the archon handler refuses a .rad below the env's size floor" {
truncate -s 1048576 "$STAGE/models/payload.rad" # 1 MiB, far under the floor
archon_payload_env
export STUB_IMAGE="$ARCHON_IMAGE"
run_handler dankserver frank-archon
[ "$status" -ne 0 ]
[[ "$output" == *"frank-archon: .rad: is 1 MiB, expected >= "* ]]
[[ "$output" == *"convert-swift15-w4dn8.sh"* ]]
[ "$(restarts_requested)" = "0" ]
}
@test "the archon handler clears the legacy container identities and relinks the unit" {
# The retired identities (the R9V container, the ad hoc probe, the glm
# experiment) hold the same GPUs, so a deploy that leaves one running
# leaves the new engine unable to bind. The unit link is the root-unit
# shape's whole install.
archon_payload_env 2
export STUB_IMAGE="$ARCHON_IMAGE"
run_handler dankserver frank-archon
[ "$status" -eq 0 ]
grep -q 'restart frank-archon.service' "$STAGE/systemctl.log"
[ -L "$STAGE/systemd/frank-archon.service" ]
local ctr
for ctr in frank-archon frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe; do
grep -qx "rm -f $ctr" "$STAGE/podman.log"
done
}
@test "no LLM handler reports green when its unit has no MainPID behind the port" {
# The 2026-09-26 false green: a leftover probe held :7770 and satisfied
# the health curl while the real unit was stopped.
archon_payload_env 2
export STUB_IMAGE="$ARCHON_IMAGE"
export STUB_MAIN_PID=0
run_handler dankserver frank-archon
[ "$status" -ne 0 ]
[[ "$output" == *"frank-archon.service has no MainPID"* ]]
}
@test "a refused LLM handler leaves its service pending in the deploy state" {
export STUB_IMAGE=
run_handler dankserver frank-archon
[ "$status" -ne 0 ]
grep -q 'status=pending' "$STAGE/state/frank-archon"
[ ! -e "$STAGE/state/frank-zealot" ]
}
+172
View File
@@ -0,0 +1,172 @@
#!/usr/bin/env bats
# Seam: the shared artifact gates every LLM endpoint runs before it starts.
#
# The rule these encode: a serve script and the deployer must refuse the
# SAME missing/truncated artifact, and they must refuse it the same way.
# Two hand-written copies of the same gates drift apart until a deploy
# passes what the unit would refuse; the library is the one place the
# refusal lives, and both callers source it.
#
# Everything here is observable: arguments in, exit status and stderr out.
# Nothing inspects the library's internals.
load helpers
LIB="$REPO_ROOT/scripts/lib/llm-gates.sh"
setup() {
TMP="$(mktemp -d)"
# A fixture with a known, independent size: 12 bytes.
printf '0123456789ab' > "$TMP/artifact.bin"
mkdir -p "$TMP/flat"
touch "$TMP/flat/tokenizer.json" "$TMP/flat/config.json"
}
teardown() {
rm -rf "$TMP"
}
# call <invocation> — run one gate in a fresh shell under `set -e`, exactly
# as a serve script sources the library, and capture status + both streams.
call() {
run env GATE_PREFIX="${GATE_PREFIX:-testgate}" PATH="$PATH" \
bash -c "set -euo pipefail; source '$LIB'; $1"
}
@test "a present artifact passes the file gate with no output" {
GATE_PREFIX=testgate call "require_file '$TMP/artifact.bin' checkpoint 'fetch it'"
[ "$status" -eq 0 ]
[ -z "$output" ]
}
@test "an absent artifact fails the file gate naming the path and the remedy" {
call "require_file '$TMP/absent.bin' checkpoint 'run scripts/download-frank-archon-swift15.sh'"
[ "$status" -ne 0 ]
[[ "$output" == *"checkpoint: missing"* ]]
[[ "$output" == *"$TMP/absent.bin"* ]]
[[ "$output" == *"run scripts/download-frank-archon-swift15.sh"* ]]
}
@test "the file gate refuses a directory where an artifact is required" {
# The serve gate is `-f`, not `-e`: a directory named like the payload
# (an unpacked staging dir) must not boot the engine.
call "require_file '$TMP/flat' checkpoint 'fetch it'"
[ "$status" -ne 0 ]
[[ "$output" == *"checkpoint"* ]]
}
@test "a symlinked artifact is refused: a HF snapshot link dangles in the container mount" {
ln -s "$TMP/artifact.bin" "$TMP/linked.bin"
call "require_file '$TMP/linked.bin' checkpoint 'fetch it'"
[ "$status" -ne 0 ]
[[ "$output" == *"checkpoint"*"missing"* ]]
}
@test "the byte gate accepts the exact contracted size" {
call "require_bytes '$TMP/artifact.bin' checkpoint 12 'fetch it'"
[ "$status" -eq 0 ]
[ -z "$output" ]
}
@test "the byte gate refuses a size mismatch quoting both the real and the expected size" {
# The Hub's declared byte count is the contract; curl writes the error
# body and exits 0, so a truncation is invisible without this gate.
call "require_bytes '$TMP/artifact.bin' checkpoint 42 'run the fetch script'"
[ "$status" -ne 0 ]
[[ "$output" == *"12 bytes"* ]]
[[ "$output" == *"expected 42"* ]]
[[ "$output" == *"run the fetch script"* ]]
}
@test "the byte gate is an equality contract, not a floor" {
# A LARGER file is not a better artifact: it is a different artifact
# (the compacted r2 of a checkpoint is byte-exact, not "at least").
call "require_bytes '$TMP/artifact.bin' checkpoint 6 'fetch it'"
[ "$status" -ne 0 ]
[[ "$output" == *"expected 6"* ]]
}
@test "the minimum-size gate accepts the threshold itself" {
dd if=/dev/zero of="$TMP/two-mib.rad" bs=1048576 count=2 status=none
call "require_min_mib '$TMP/two-mib.rad' container 2 'rebuild it'"
[ "$status" -eq 0 ]
[ -z "$output" ]
}
@test "the minimum-size gate refuses below the threshold naming both sizes" {
dd if=/dev/zero of="$TMP/one-mib.rad" bs=1048576 count=1 status=none
call "require_min_mib '$TMP/one-mib.rad' container 2 'rebuild it'"
[ "$status" -ne 0 ]
[[ "$output" == *"1 MiB"* ]]
[[ "$output" == *"2 MiB"* ]]
[[ "$output" == *"rebuild it"* ]]
}
@test "the flat-directory gate accepts a directory of regular files" {
call "require_dir_files '$TMP/flat' tokenizer 'fetch the tokenizer' tokenizer.json config.json"
[ "$status" -eq 0 ]
[ -z "$output" ]
}
@test "the flat-directory gate names the member that is absent" {
call "require_dir_files '$TMP/flat' tokenizer 'fetch the tokenizer' tokenizer.json vocab.json"
[ "$status" -ne 0 ]
[[ "$output" == *"tokenizer"*"missing"* ]]
[[ "$output" == *"vocab.json"* ]]
}
@test "the flat-directory gate refuses a symlinked member" {
# A HuggingFace cache snapshot's entries are symlinks into blobs/ and
# dangle inside the container mount even when the directory is real.
ln -s /nowhere/x "$TMP/flat/vocab.json"
call "require_dir_files '$TMP/flat' tokenizer 'fetch the tokenizer' tokenizer.json vocab.json"
[ "$status" -ne 0 ]
[[ "$output" == *"vocab.json"*"missing or a symlink"* ]]
}
@test "the flat-directory gate refuses a missing directory" {
call "require_dir_files '$TMP/nope' tokenizer 'fetch the tokenizer' tokenizer.json"
[ "$status" -ne 0 ]
[[ "$output" == *"tokenizer: missing"* ]]
}
@test "the flat-directory gate refuses a symlinked directory" {
# The snapshot directory itself can be the link (a HF cache blobs/
# sibling), and a link into the container mount dangles there too.
ln -s "$TMP/flat" "$TMP/linked-dir"
call "require_dir_files '$TMP/linked-dir' tokenizer 'fetch the tokenizer' tokenizer.json"
[ "$status" -ne 0 ]
[[ "$output" == *"tokenizer: missing"* ]]
}
@test "the image gate refuses a ref the storage does not hold" {
# A name@sha256 ref exists only if THAT manifest was pulled, so the
# gate is the ref itself — one command, no grep race.
skip_if_no_podman
call "require_image 'example.invalid/nothing@sha256:0000000000000000000000000000000000000000000000000000000000000000' 'podman pull it'"
[ "$status" -ne 0 ]
[[ "$output" == *"not in podman storage"* ]]
[[ "$output" == *"podman pull it"* ]]
}
@test "every failure names the endpoint the operator is standing in front of" {
GATE_PREFIX=frank-dragoon call "require_file '$TMP/absent.bin' checkpoint 'fetch it'"
[ "$status" -ne 0 ]
[[ "$output" == frank-dragoon:* ]]
}
@test "a failing gate stops the caller from reaching the runtime" {
# The composition property that matters: the gate is not advisory. A
# serve script that calls it and then execs the engine must die first.
cat > "$TMP/serve-like.sh" <<EOF
#!/usr/bin/env bash
set -euo pipefail
source '$LIB'
require_file '$TMP/absent.bin' checkpoint 'fetch it'
echo RUNTIME-EXECUTED
EOF
run bash "$TMP/serve-like.sh"
[ "$status" -ne 0 ]
[[ "$output" != *"RUNTIME-EXECUTED"* ]]
[[ "$output" == *"checkpoint: missing"* ]]
}
+35
View File
@@ -80,3 +80,38 @@ unit_repo_path() {
}
done < <(find "$REPO_ROOT/scripts" -name "frank-*.service" -type f | grep -v AGENTTRASH | sort)
}
@test "every LLM root unit is self-healing against a voluntary exit" {
# Restart=always, not on-failure. Every endpoint here is a container
# runner (podman run) or a server whose front end can exit 0 on a
# control-plane event; systemd treats exit 0 as a CLEAN stop and
# on-failure leaves the endpoint dark until the next boot, with the
# unit reporting success. No LLM unit may ship the weaker policy: a
# single endpoint on on-failure is a daemon that goes dark silently.
local unit
while IFS= read -r unit; do
grep -qx 'Restart=always' "$unit" || {
echo "$(basename "$unit"): $(grep '^Restart=' "$unit") (must be always)"
return 1
}
done < <(find "$REPO_ROOT/scripts" -name "frank-*.service" -type f | grep -v AGENTTRASH | sort)
}
@test "every LLM root unit restarts promptly and starts at boot" {
# RestartSec=10 is the fleet's cadence: fast enough that a transient
# fault is invisible to traffic, slow enough that a hard fault does not
# spin the engine (each radiance boot re-pins ~147 GiB).
# WantedBy=multi-user.target is boot persistence for a ROOT unit —
# default.target is the USER-manager's target and means nothing here.
local unit
while IFS= read -r unit; do
grep -qE '^RestartSec=([1-9]|1[0-9]|20)$' "$unit" || {
echo "$(basename "$unit"): no RestartSec in 1..20"
return 1
}
grep -qx 'WantedBy=multi-user.target' "$unit" || {
echo "$(basename "$unit"): $(grep '^WantedBy=' "$unit")"
return 1
}
done < <(find "$REPO_ROOT/scripts" -name "frank-*.service" -type f | grep -v AGENTTRASH | sort)
}
+2 -2
View File
@@ -112,11 +112,11 @@ GLUETUN="$REPO_ROOT/dankclaw/searxng/quadlet/gluetun.container"
# Public edge: the vhost must proxy to the challenge proxy, not straight to the app.
awk '
/^marreta\.franksdank\.cloud \{/ { inblk=1 }
inblk && /reverse_proxy anubis:8923/ { challenge=1 }
inblk && /import guarded/ { challenge=1 }
inblk && /^\}/ { inblk=0 }
END { exit challenge ? 0 : 1 }
' "$REPO_ROOT/dankclaw/Caddyfile" \
|| { echo "marreta vhost must reverse_proxy anubis:8923"; return 1; }
|| { echo "marreta vhost must import the guarded chain (the one named Anubis snippet)"; return 1; }
}
@test "the Caddyfile post-auth listener routes marreta to the published 8082" {
+8 -5
View File
@@ -433,11 +433,14 @@ assert any('ALERTS' in e for e in exprs), exprs
grep -q -- '--web.listen-address=127.0.0.1:9100' "$unit"
grep -q -- '--web.listen-address=192.168.0.235:9100' "$unit"
! grep -qE -- '--web\.listen-address=(0\.0\.0\.0|:9100|\[::\])' "$unit"
# One owner per container name: the compose service must be gone, or two
# definitions fight over the same container.
! grep -qE '^\s+node-exporter:' "$REPO_ROOT/dankserver/compose.yaml"
# The deploy path has to know about it on this target.
grep -q 'dankserver:node-exporter' "$REPO_ROOT/scripts/deploy-remote.sh"
# One owner per container name: Dankserver has no compose file at all
# since the 2026-10-13 retirement (it lived beside diverged quadlet
# mounts for weeks — the two-owners trap by construction).
[ ! -e "$REPO_ROOT/dankserver/compose.yaml" ]
# The deploy path has to know about it on this target — acceptance
# is the inventory (rank 3), not a literal list in the runner.
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankserver --service node-exporter
[ "$status" -eq 0 ]
run python3 "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver \
--paths dankserver/observability/quadlet/node-exporter.container
[ "$status" -eq 0 ]
+138
View File
@@ -0,0 +1,138 @@
#!/usr/bin/env bats
# Seam: the service inventory (rank 3). shared/services.yml is the one
# inventory of deployable services; scripts/service-map.py is its only
# reader, and every consumer (deploy-plan.py, deploy-remote.sh's
# allow-list, deploy.sh's --service gate, test-changed.sh's seam mapper,
# the Pi-hole hosts render, the Prometheus scrape facts, the homepage
# probes) goes through it. These tests pin the inventory's integrity,
# the planner's selection against a captured fixture, and the rendered
# artifacts' drift against the tracked copies.
load helpers
@test "inventory: schema, unit claims, path claims, compose ownership, and render drift are sound" {
run python3 "$REPO_ROOT/scripts/service-map.py" check
[ "$status" -eq 0 ]
}
@test "inventory: the deployers' allow-list is the inventory" {
# Every service the inventory places on a host must pass "allows" for
# that host, and a name the inventory never places must be refused.
for host in dankclaw dankserver frankbook-pro; do
while IFS= read -r svc; do
[ -n "$svc" ] || continue
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host "$host" --service "$svc"
[ "$status" -eq 0 ] || { echo "allows refused $host:$svc"; return 1; }
done < <(python3 "$REPO_ROOT/scripts/service-map.py" services --host "$host")
done
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankserver --service sonarr
[ "$status" -ne 0 ] # the *arr stack lives on frankbook-pro, not dankserver
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankclaw --service frank-archon
[ "$status" -ne 0 ] # frank-archon lives on dankserver
}
@test "inventory: the planner reproduces the pinned selection fixture" {
# tests/fixtures/service-plan-golden.tsv is the characterization of
# path -> service selection across every tracked path, every target,
# and the multi-path combos the old hand-written planner was fed.
python3 - "$REPO_ROOT" <<'PYEOF'
import importlib.util
import sys
root = sys.argv[1]
spec = importlib.util.spec_from_file_location("sm", root + "/scripts/service-map.py")
sm = importlib.util.module_from_spec(spec)
spec.loader.exec_module(sm)
inv = sm.Inventory.load(root + "/shared/services.yml")
bad = []
with open(root + "/tests/fixtures/service-plan-golden.tsv") as fh:
for line in fh:
host, paths, want = line.rstrip("\n").split("\t")
got = ",".join(sm.select(inv, host, paths.split("+")))
if got != want:
bad.append(f"{host} {paths}\n want: {want}\n got: {got}")
if bad:
print(f"{len(bad)} selection mismatches:", *bad[:20], sep="\n")
sys.exit(1)
PYEOF
}
@test "inventory: deploy-plan.py keeps its CLI contract on top of service-map" {
# stdin paths, positional paths, and the --repo/--base/--head diff mode
# all route to the same selection.
run bash -c "cd '$REPO_ROOT' && printf '%s\n' dankclaw/searxng/quadlet/gluetun.container | \
scripts/deploy-plan.py --target dankclaw"
[ "$status" -eq 0 ]
[ "${lines[0]}" = "gluetun" ]
run "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver scripts/frank-archon.env
[ "$status" -eq 0 ]
[ "${lines[0]}" = "frank-archon" ]
run "$REPO_ROOT/scripts/deploy-plan.py" --target dankclaw --repo "$REPO_ROOT" --base HEAD --head HEAD
[ "$status" -eq 0 ]
[ -z "$output" ]
}
@test "inventory: the Pi-hole hosts file is a rendered artifact" {
# The tracked addn-hosts must be byte-identical to the render; editing
# it by hand is drift the inventory check also catches.
run bash -c "cd '$REPO_ROOT' && python3 scripts/service-map.py render pihole-hosts | \
diff -u dankclaw/pihole/addn-hosts -"
[ "$status" -eq 0 ]
}
@test "inventory: changed-test seams for a service unit are the service's own" {
# The rank-3 contract: a unit change selects the OWNING service's
# declared seams, not the old shotgun of unrelated quadlet seams.
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths dankclaw/gitea/quadlet/gitea.network
[ "$status" -eq 0 ]
[[ "$output" == *"tests/caddyfile.bats"* ]]
[[ "$output" == *"tests/deployment_scoping.bats"* ]]
[[ "$output" != *"tests/open_webui_quadlet.bats"* ]]
[[ "$output" != *"tests/searxng_quadlet.bats"* ]]
}
@test "inventory: every tracked quadlet unit is claimed by exactly one service" {
# check units covers claim conflicts; this pins the census size so a
# unit added WITHOUT an inventory entry (unclaimed) fails the seam.
python3 - "$REPO_ROOT" <<'PYEOF'
import importlib.util
import subprocess
import sys
from pathlib import PurePosixPath
root = sys.argv[1]
spec = importlib.util.spec_from_file_location("sm", root + "/scripts/service-map.py")
sm = importlib.util.module_from_spec(spec)
spec.loader.exec_module(sm)
inv = sm.Inventory.load(root + "/shared/services.yml")
graph = sm.graph_for(tuple(inv.hosts))
owned = {u for u in graph.units}
problems = []
tracked_units = [
p for p in sm.tracked_files()
if PurePosixPath(p).name.endswith((".container", ".network")) and not sm.is_trash(p)
]
for path in tracked_units:
parts = PurePosixPath(path).parts
name = parts[-1]
key = (parts[0], name.rsplit(".", 1)[0])
if key not in graph.units and key not in graph.networks:
problems.append(f"unclaimed unit-file path: {path}")
if problems:
print("\n".join(problems))
sys.exit(1)
PYEOF
}
@test "inventory: the node-exporter unit is a per-host rendered template" {
# Both quadlets are rendered artifacts: the render — not a hand-edited
# mirror kept in step by a comment — defines the unit on every host.
for host in dankclaw dankserver; do
run bash -c "cd '$REPO_ROOT' && python3 scripts/service-map.py render node-exporter-unit --host $host | \
diff -u $host/observability/quadlet/node-exporter.container -"
[ "$status" -eq 0 ] || { echo "$host unit drift"; return 1; }
done
# Hand-edited copies fail the drift gate:
grep -q "Keep this file in step" dankserver/observability/quadlet/node-exporter.container && return 1
true
}
+2
View File
@@ -0,0 +1,2 @@
#!/usr/bin/env bash
exit 0
+11
View File
@@ -0,0 +1,11 @@
#!/usr/bin/env bash
# Stub podman for the deploy-handler seam: `image exists` succeeds only for
# $STUB_IMAGE (empty = nothing in storage); every other verb is a no-op.
# The invocations are logged so a test can read what the handler executed.
printf '%s\n' "$*" >> "${STUB_PODMAN_LOG:-/dev/null}"
case "$1 $2" in
"image exists")
[[ -n "${STUB_IMAGE:-}" && "$3" == "$STUB_IMAGE" ]] || exit 1
;;
esac
exit 0
+2
View File
@@ -0,0 +1,2 @@
#!/usr/bin/env bash
exit 0
Executable
+2
View File
@@ -0,0 +1,2 @@
#!/usr/bin/env bash
exit 0
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
# Stub sudo for the deploy-handler seam: run_privileged pipes the password
# on stdin and calls `sudo -S -- <cmd>`. The stub drops the password and the
# flags and execs the command, so no privilege gate hides a handler bug.
cat >/dev/null 2>&1 || true
while [ "$#" -gt 0 ]; do
case "$1" in
-S|--prompt) shift ;;
--) shift; break ;;
*) break ;;
esac
done
exec "$@"
+12
View File
@@ -0,0 +1,12 @@
#!/usr/bin/env bash
# Stub systemctl for the deploy-handler seam: every invocation is logged, so
# a test asserts on what the handler ASKED systemd to do. `is-active` answers
# "inactive" (the state a cutover finds the unit in) and MainPID answers
# $STUB_MAIN_PID — 0 is the stray-port-holder shape, the false-green gate
# every LLM handler reads after a health answer.
printf '%s\n' "$*" >> "${STUB_SYSTEMCTL_LOG:-/dev/null}"
case "$*" in
"is-active "*) echo inactive ;;
*"MainPID --value "*) echo "${STUB_MAIN_PID:-4242}" ;;
esac
exit 0